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Executive  Summary 


The  Human  Interoperability  (HI)  initiative  is  intended  to  improve  the  responsiveness,  effieieney, 
and  effeetiveness  of  organizations  when  they  partner  externally  or  internally.  This  eapability 
should  improve  the  ability  of  organizations  to  exeeute  missions  aeross  the  eontinuum  from 
normal  and  routine  (sueh  as  Maritime  Domain  Awareness  and  maintaining  port  seeurity)  to 
national  erises  (sueh  as  major  terrorist  attaeks). 

Human  Interoperability  researeh  beginning  in  FY  08  and  eontinuing  in  future  years  will  produee 
three  produets:  (1)  a  struetured,  multi-diseiplinary  method  for  studying  inter-  and  intra¬ 
organization  eollaboration  at  the  operational  level,  and  a  taxonomy  of  barriers  and  solution 
strategies;  (2)  a  speeifieation  of  barriers  to  Human  Interoperability  in  DoD  and  its  partners;  and 
(3)  tools  for  organization  designers  and  leaders:  ease  studies,  guidelines,  measures,  and  models 
that  support  partnership  planning. 

This  doeument  reports  findings  from  initial  researeh,  led  by  OSD  Nil,  to  explore  the  diseipline 
and  dimensions  of  HI.  The  goals  of  the  researeh  team  were  to  develop  theory  eoneeming  HI, 
eapture  and  analyze  initial  evidenee  of  barriers  to  organizational  eollaboration,  and  to  outline 
objeetives  for  future  researeh. 

The  team’s  initial  approaeh  to  a  theoretieal  framework  of  human  interoperability  between  large 
organizations  and  organizational  partnerships  foeused  on  organizations  as  eomplex  systems.  In 
this  view,  organizations  shape,  and  are  shaped  bi-direetionally,  by  their  environments. 
Organizations  employ  a  eatalog  of  struetures  (personal  relationships,  liaisons,  joint  task  forees, 
information  sharing  and  systems,  ete.)  through 
whieh  they  eoordinate  and  exeeute  work,  and 
trade  valued  produets,  sueh  as  information. 

Several  meehanisms  -  poliey,  proeesses, 
eultural  faetors,  information  systems,  and 
individual  eompeteney  -  modulate  the  type, 
method,  speed,  volume,  and  quality  of  aetivity 
over  these  struetures.  Astute  organizations  adapt 
struetures  and  eontrol  meehanisms  over  time. 

They  eannot  reliably  do  this  by  ehoosing 
between  struetures  and  eontrols  that  are 
engineered  into  the  organization  a  priori, 
beeause  organizations  typieally  are  not  designed 
for  adaptation,  nor  would  they  be  effieient  or  effeetive  on  any  given  mission  if  they  were  erafted 
to  satisfy  all  possible  future  missions.  Rather,  the  eomplex  systems  view  leads  us  to  postulate 
that  organizations,  like  their  individual  members,  develop  an  adaptive  eompetenee  to  form,  use, 
and  evolve  struetures  and  eontrols  that  improve  inter-  and  intra-organizational  performanee.  The 
nature  of  these  eompeteneies  is  a  focus  of  proposed  research,  as  is  the  catalog  of  structures  and 
controls  that  organizations  develop. 

The  first  experimental  HI  event,  HI  08,  took  place  in  June  2008.  OSD  Nil  assembled  a  research 
team  to  conduct  this  initial  investigation  of  Human  Interoperability  issues.  The  team  executed  its 
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study  during  Trident  Warrior  08  (TW  08,  a  NAVNETWARCOM  sponsored  field  experiment)  in 
June  2008.  The  data  eolleeted  in  interviews  informed  our  understanding  of  many  of  the 
struetures  and  eontrollers  refereneed  above.  We  present  a  summary  of  these  findings  and  of  their 
implications  for  the  design  and  conduct  of  organizational  partnerships  on  page  18  in  Table  1. 

We  recommend  that  a  program  of  research  be  developed  to  significantly  deepen  and  broaden  our 
understanding  of  Human  Interoperability  barriers  and  solutions  for  DoD  organizations.  This 
work  will  produce  refined  methods  for  studying  this  and  other  complex  fields  of  human 
interaction,  and  it  will  lay  the  foundation  on  which  to  develop  products  for  training  and  aiding 
leaders  as  they  develop  partnerships  that  strengthen  their  organizations  and  accomplish  missions. 
Critical  to  that  research  is  the  definition  of  objectives  and  questions  it  must  address.  We  have 
drafted  those  products,  and  describe  how  they  can  be  put  to  use  on  page  21 . 

The  authors  welcome  comments  and  recommendations  concerning  this  work.  Please  contact  Dr. 
Shelley  Gallup,  Naval  Postgraduate  School,  spgallup@nps.edu,  or  Dr.  Jared  Freeman,  Aptima, 
freeman@aptima.com. 


Introduction 

Human  Interoperability:  Its  Function,  Form,  and  Future 

The  Human  Interoperability  (HI)  research  initiative  is  intended  to  improve  performance  within 
organizations  and  between  them.  More  specifically,  HI  research  is  intended  to  improve  the 
responsiveness,  efficiency,  and  effectiveness  of  organizations  that  must  partner  or  dynamically 
integrate  elements  of  their  organization  (people,  processes,  resources,  authorities, 
responsibilities)  with  those  of  others  in  periods  of  unique  and  possibly  disorienting  need.  This 
improved  adaptability  of  organizations  is  theorized  to  also  increase  their  ability  to  execute 
missions  across  the  continuum  from  normal  and  routine  (such  as  Maritime  Domain  Awareness 
and  maintaining  port  security)  to  national  crises  (such  as  major  terrorist  attacks). 

Barriers  to  effective  cross-organization  collaboration  are  increasingly  a  subject  of  concern  as 
DoD  and  its  partners  attempt  to  coordinate  activity  at  multiple  levels  during  unique  operations 
that  bring  together  new  combinations  of  organizations  to  address  a  broad  spectrum  of  needs.  This 
is  precisely  the  challenge  facing  the  Department  of  Homeland  Security;  DHS  has  struggled  and 
sometimes  failed  to  integrate  its  seven  operational  units'  (Hocevar,  Thomas,  and  Jansen,  2006). 
The  HI  research  initiative  attempts  to  (1)  to  develop  a  structured,  multi-disciplinary  approach  to 
analyzing  collaboration,  co-operation,  and  coordination  success  and  failures  at  the  operational 
level;  (2)  identify  and  categorize  barriers  to  these  activities;  (3)  and  design,  develop,  and  test 
solutions  that  overcome  these  barriers.  To  accomplish  this,  the  HI  initiative  must  develop  theory, 
experimental  methods,  measures,  and  experiments  that  grow  knowledge  of  human 
interoperability,  as  well  as  products  of  direct  benefit  to  organizations:  dynamic  models  to  support 
HI  diagnosis  and  design,  guidance  to  leaders,  exemplars  that  motivate  improvement. 

In  somewhat  greater  detail,  the  HI  research  team  aims  to  develop: 

•  Experimentation  —  Conduct  an  initial  study,  piggybacking  the  resources  of  Trident 
Warrior  '08,  that  surfaces  barriers  to  HI.  Conduct  more  structured  and  rigorous 
experiments  beginning  in  FY09. 

•  Methodology  —  Define  an  efficient  and  productive  method  for  studying  HI  in 
experiments. 

•  Theory  —  Define  a  theory  that  enumerates  the  types  of  human  interfaces  and  protocols 
that  enable  effective  collaboration  between  organizations.  Test  and  refine  this  theory 
through  experimentation. 

•  Measures  —  Define  measures  of  human  interoperability  between  enterprises,  and  of  the 
effectiveness  of  interfaces  that  support  interoperability. 

•  Exemplars  -  Identify,  foster,  and  publicize  examples  of  HI,  organizational  interfaces  that 
the  community  can  inspect  and  replicate  to  improve  the  enterprise. 


*  The  operational  entities  within  DHS  are  Transportation  Security  Administration,  U.S.  Customs  and  Border 
Protection,  U.S.  Citizenship  and  Immigration  Services,  U.S.  Immigrations  Customs  Enforcement,  U.S.  Secret 
Service,  the  Federal  Emergency  Management  Agency,  and  the  U.S.  Coast  Guard. 
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•  Management  resources  -  Develop  guidance  that  helps  leaders  efficiently,  reliably,  and 
measurably  improve  HI.  This  guidance  will  be  identified  through  sensitivity  analyses 
conducted  using  the  modeling  technology,  above. 

•  Organizational  design  resources  -  Apply  systems  engineering  principles  to  define  design 
elements  -  structures,  processes,  training,  etc.  -  that  increase  the  potential  for  human 
interoperability. 

•  Model  -  Define  formal  models  of  HI  that  support  reliable  diagnosis  of  causes  of  failures 
of  HI  and  prediction  of  intended  and  unintended  effects  of  actions  to  improve  HI. 
Implement  this  model  in  a  software  tool  that  aids  leaders  of  organizations,  as  well  as 
researchers  who  must  test  new  hypotheses  about  HI  and  methods  of  managing  it. 

The  first  HI  experimentation,  HI08,  took  place  in  June  2008.  OSD  Nil  assembled  a  research 
team  to  conduct  this  initial  investigation,  under  the  direction  of  Stefania  Brown-VanHoozer, 
Ph.D.  The  team  executed  its  study  during  Trident  Warrior  08  (TW  08),  a  field  experiment 
sponsored  by  NAVNETWARCOM.  Data  collection  in  HI08  focused  on  three  dimensions  of  HI: 

(1)  human  infrastructures  (personal  relationships,  liaisons,  joint  centers,  etc.)  that  people 
establish  to  coordinate  their  work  with  other  humans  -  person-to-person,  person-to- 
organizations,  person  (s)  to  systems; 

(2)  factors  that  control  the  volume  and  quality  of  work  executed  through  this  infrastructure, 
e.g.,  policy,  processes,  social-cultural  nuances,  cognitive  issues,  human  interaction  with 
information  systems,  authoritative  lanes,  and  other  factors  of  human  interactions; 

(3)  sharing  of  information 

Findings  from  that  effort  are  reported  here,  as  are  recommendations  concerning  future  HI 
experimentation.  We  begin,  however,  by  presenting  a  draft  theory  of  HI  to  help  the  reader 
understand  the  orientation  of  this  effort  and  the  structure  of  the  analyses  reported  below. 


Toward  a  theory  of  Human  Interoperability 

This  section  of  the  report  presents  a  draft  theory  of  Human  Interoperability.  The  theory  serves 
several  functions.  First,  it  clarifies  what  we  mean  when  we  talk  about  Human  Interoperability. 
Second,  it  familiarizes  the  reader  with  the  structure  used  to  categorize  and  analyze  findings  from 
HI08  (below).  Third,  the  theory  provides  the  research  team  with  considerable  leverage  on  the 
problem  of  defining  research  objectives  for  future  Human  Interoperability  studies. 

A  variety  of  research  disciplines  address  questions  concerning  how  organizations  function. 
Organizational  psychology  explores  characteristics  of  team  members  and  team  interactions  that 
predict  effectiveness.  Systems  engineering  methods  such  as  the  DoD  Architectural  Framework 
(Department  of  Defense,  2007)  specify  operational  aspects  of  socio-technical  systems  (  “OV” 
diagrams  of  organization  function,  connectivity,  information  dependencies,  roles  and 
relationships,  activities,  and  states),  technical  aspects  of  these  system  (“SV”  diagrams  of  entity 
interfaces,  functions,  activities,  performance  parameters,  technologies,  information  formats,  and 
others),  and  standards  employed  by  the  system  (“TV”  diagrams).  Cultural  research  defines 
attributes  of  culture  (e.g.,  power  distance,  individualism,  etc.)  that  differ  reliably  between 
national  or  ethnic  groups  and  that  can  help  predict  attitudes  and  behaviors  (For  a  concise  review 
of  these  literatures,  see  Appendix  A:  Fiterature  Review). 
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None  of  these  approaehes  provides  the  diagnostic  or  predictive  power  needed  to  understand  and 
shape  interactions  within  and  between  organizations  at  the  operational  level.  It  is  at  this  level  that 
organizations  systematically  create,  use,  and  evolve  internal  and  external  partnerships.  In 
particular,  the  system  engineering  methods  on  which  DoD  heavily  relies  (e.g.,  the  DoD 
Architectural  Framework,  2007)  were  not  designed  to  define  how  humans  within  organizations 
interact;  they  were  not  designed  to  model  highly  adaptive  organizational  structures  and 
processes;  and  they  are  typically  applied  to  model  the  integration  of  entities  within  one 
organization,  not  the  actual  or  potential  interoperability  between  organizations.  In  short,  existing 
engineering  solutions  represent  single,  static,  organizations,  and  not  human  activity  within 
organizations  that  collaborate  dynamically  and  opportunistically. 

Interactions  at  the  human-human  interface  are,  however,  of  increasing  concern  as  the  scope  and 
complexity  of  missions  grows  -  from  warfighting  to  hybrid  missions  involving  peacekeeping, 
political  change,  humanitarian  assistance,  nation  building,  infrastructure  restitution,  law 
enforcement,  and  other  activities.  In  these  cases,  units  within  organizations  and  organizations 
themselves  often  must  ally  and  collaborate  in  new  ways  to  define  and  resolve  problems.  Further, 
these  complex  collaborations  often  must  create  or  extend  a  capability  for  sharing  information  in 
order  to  achieve  the  close  coupling  and  synchronization  that  missions  demand. 

However,  it  is  often  difficult  for  units  and  organizations  to  join  forces,  despite  ubiquitous 
information  sharing  technologies  (e.g.,  web  browsers,  common  databases,  teleconferencing,  chat, 
email).  Barriers  of  several  types  often  afflict  the  effectiveness  and  efficiency  with  which 
organizational  entities  interact. 

•  Policy  barriers  may  prevent  institutions  from  collaborating  effectively.  For  example, 
non-govemmental  organizations  (NGOs)  often  have  policies  that  prohibit  interaction  with 
DoD,  even  when  DoD  offers  material  aid  of  high  value  to  NGO  missions.  Legal  barriers 
(e.g.,  the  Health  Insurance  Portability  and  Accountability  Act  of  1996)  prohibit  health 
care  providers  from  sharing  certain  information  about  patients,  and  the  Posse  Comitatus 
Act  (1878)  prohibits  DoD  from  participating  in  police  actions  on  U.S.  soil. 

•  Structural  barriers  may  effectively  disable  the  very  interfaces  through  which  interaction 
should  occur.  For  example,  an  element  of  one  organization  may  have  no  corresponding  or 
effective  element  in  another  organization. 

•  Process  barriers  occur  when  there  are  no  procedures  for  sharing  information,  decisions, 
and  tasks  across  organizations,  or  when  the  procedures  of  one  organization  are  obscure  or 
even  infeasible  (e.g.,  with  respect  to  security)  for  another. 

•  Data  interoperability  barriers  exist  when  the  information  products  of  one  institution 
have  no  relationship  to  the  information  input  requirements  of  another,  when  systems  with 
potential  information  value  cannot  be  used  by  a  given  organization,  or  when  the  language 
used  in  one  organization  is  ambiguous  or  conflicts  with  the  language  used  at  another. 

•  Cultural  barriers  exist  when  authority,  reward  structures,  tolerance  for  uncertainty,  time 
orientation,  and  other  constructs  (Hofstede,  1980,  1994)  are  in  conflict.  For  example,  the 
military  command  hierarchy  often  is  seen  as  in  conflict  with  the  relatively  flat  and 
adaptive  (or  flexible)  authority  structures  of  law  enforcement  and  civilian  organizations. 

•  Cognitive  and  affective  barriers  exist  when  an  individual  cannot  effectively  support 
collaboration  between  organizations  because  they  do  not  have  the  knowledge  and  skills  to 
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do  so,  or  when  they  fail  to  develop  trust  and  other  requirements  of  relationships 

effieiently  and  effeetively. 

The  length  of  this  list  may  give  the  reader  some  sense  of  the  eomplexity  of  HI.  Existing 
approaches  to  the  analysis  and  design  of  organizations  (see  Appendix  A:  Literature  Review)  do 
not  address  many  of  these  aspects  of  complexity,  nor  do  they  confront  this  fact:  organizations 
evolve  with  their  missions,  their  interactions  with  other  organizations,  and  the  initiatives  and 
ambitions  of  their  members.  Recent  literature  addresses  these  issues  of  evolution  in  part  by 
viewing  organizations  as  complex  systems.  Marion  and  Uhl-Bien  (2002)  approach  organizations 
as  networks  of  agents  seeking  to  fulfill  their  individual  and  collective  requirements.  While  this 
approach  allows  for  the  creation  of  intentionally  designed  organizations,  it  also  accounts  for 
spontaneous  emergence  of  organizations  and  partnerships,  and  for  behaviors  in  organizations  that 
reflect  local  requirements  (and  even  may  conflict  with  global,  organizational  requirements).  The 
theory  accounts  for  the  dynamic  response  of  organizations  in  complex  and  uncertain 
environments  given  their  state. 

In  practice,  organizational  adaptation  is  often  slow,  difficult,  and  somewhat  unpredictable.  Thus, 
adaptation  often  cannot  be  completed  rapidly  enough  for  time  critical  missions,  nor  can  leaders 
have  confidence  that  adaptation  will  improve  execution.  These  problems  have  been  found  in 
laboratory  experiments,  as  well  as  real  world  observational  studies.  Human-team-in-the-loop 
experiments  showed  that  organizations  can  adapt  their  structure  (Entin,  Weil,  Kleinman, 
Hutchins,  Hocevar,  Kemple,  and  Serfaty,  2004)  to  fit  mission  demands.  However,  the  effort 
required  to  shift  form  is  great,  and  thus  the  probability  of  successful  adaptation  is  low.  Eurther, 
the  amount  of  effort  required  to  change  structure  varies  with  the  current,  structural  state  of  the 
organization.  In  experimental  studies  of  military  command  and  control  (C2),  organizations  adapt 
more  readily  from  divisional  forms  (in  which  members  control  assets  in  different  regions)  to 
functional  forms  (in  which  members  control  assets  of  different  types),  and  they  adapt  less  readily 
from  functional  to  divisional  structure  (Moon,  Hollenbeck,  Ilgen,  West,  Ellis,  Humphrey,  Porter, 
2000).  This  phenomenon  is  interpreted  in  system  dynamics  terms  as  hysteresis,  a  pattern  in 
which  change  (or  phase  transition)  is  dependent  on  history.  Relatively  little  is  known  about  how 
to  facilitate  (or  even  measure)  adaptation  between  organizational  states.  In  short,  it  is  difficult  to 
change  an  organization’s  structure  and  process,  often  unpredictably  so,  and  the  source  of 
difficulty  is  not  well  understood. 

The  perspective  that  organizations  are  complex  or  dynamical  systems  is  not  an  argument  that 
they  are  inherently  suboptimal,  unstable,  or  uncontrollable.  There  is  ample  evidence  (Levchuk, 
Meirina,  Pattipati,  and  Kleinman,  2004)  that  organizations  can  be  designed  to  fit  a  mission  or 
environment  well.  However,  it  tests  credulity  to  propose  that  an  organization  could  be  designed, 
a  priori,  to  be  optimally  effective  and  robust  to  all  possible  adaptations.  Mission-specific 
effectiveness  and  general  robustness  typically  must  be  traded  off  against  one  another.  Nor  is  it 
feasible  to  design  an  organization  with  all  of  the  interfaces  it  might  ever  need  to  interoperate  with 
potential  partners  and  to  conform  to  new  environments.  No  design  or  rules  is  likely  to 
successfully  address  the  infinite,  or  at  least  surprising,  variety  of  future  conditions. 


^  We  define  trust  here  as  eonfidenee  in  the  intentions  and  eapability  of  a  partner  to  aet  in  aeeordanee  with  one’s 
interests  and  plans. 
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By  taking  a  complex  systems  perspeetive  on  this  problem,  we  are  able  to  propose  instead  that 
organizations  aequire  and  exercise  a  competency  to  adapt^.  This  competency  should  enable 
organizations  to  invent  or  apply  structures  and  proeesses  that  are  well-suited  to  new  missions, 
particularly  when  those  missions  take  plaee  in  a  highly  unstable  and  environment  (see  Figure  1, 
from  Daft,  2000). 


STABLE 


ENVIRONMENTAL 

CHANGE 


1.  Mechanistic  structure;  formal, 
centralized 

2.  Few  departments 

3.  No  integrating  roles 

4.  Current  operations  orientation 


1.  Mechanistic  structure;  formal, 
centralized 

2.  Many  departments,  some  boundary 
spanning 

3.  Few  integrating  roles 


4.  Some  Planning 


UNSTABLE 


1.  Organic  structure,  teamwork; 
participative,  decentralized 

2.  Few  departments,  much  boundary 
spanning 

3.  Few  integrating  roles 

4.  Planning  orientation 


1.  Organic  structure,  teamwork; 
participative,  decentralized 

2.  Many  departments  differentiated, 
extensive  boundary  spanning 

3.  Many  integrating  roles 

4.  Extensive  planning,  forecasting 


SIMPLE 


ENVIRONMENTAL  COMPLEXITY 


COMPLEX 


High  adaptation  costs 
High  value  of  adaptive  competence 
under  unstable  conditions 
High  value  of  HIE 


Figure  1:  Organizations  require  an  organic  competency  to  adapt  in  unstable  and  complex 

environments. 


We  propose  an  initial  theory  of  HI  that  specifies  some  of  the  competencies  that  may  enable  an 
organization  to  evolve  its  structure  and  control  mechanisms  to  better  fit  environmental  demands. 
Figure  2  illustrates  this  framework. 


^  Adaptive  competency  is  related  to  the  notion  of  collaborative  capacity  proposed  by  Hocevar,  Thomas  and  Jansen 

(2006). 
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Adaptive  Competency 

Figure  2:  Adaptive  competency  enables  an  organization  to  adapt  its  structure  and  control 

mechanisms  to  the  environment. 


HI08  identified  a  number  of  structures  through  which  organizational  units  and  organizations 
interact  (see  Figure  3).  These  include  personal  relationships,  formal  liaisons,  joint  operation 
centers,  trusted  third  party  organizations  that  enable  each  organization  to  pass  information 
securely,  and  information  systems.  The  order  of  structures  also  reflects  declining  flexibility,  from 
individual  relationships  that  operate  quite  adaptively,  to  units  whose  policies  and  procedures  may 
constrain  interactions,  to  information  systems  that  efficiently  pass  only  the  information  they  are 
designed  to  handle. 

A  variety  of  mechanisms,  or  controllers,  modulate  the  volume,  rate,  and  quality  of  activity  that 
occurs  through  these  structures.  These  include  (see  Figure  3): 

o  Organizational  policy,  such  as  objectives  and  rewards  for  achieving  interoperability  or 
goals  that  require  it; 

o  Procedures  that  may  support  or  restrict  interoperability; 

o  Culture,  which  specifies  customs,  attitudes,  and  values  that  bear  on  collaboration; 

o  Cognitive  competencies  that  enable  individuals  to  execute  taskwork  and  teamwork 
necessary  for  interoperability;  and 

o  Data  interoperability,  which  enables  or  hinders  exchange  of  information. 
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Figure  3:  Structural  forms  and  sources  of  impedance  in  organizational  collaboration 

As  Figure  4  illustrates,  organizational  structure  and  controls  are  dynamic.  Here  time  is  laid  on 
the  vertical  axis  to  illustrate  an  evolution  of  structures  between  organizations  from  personal 
relationships  to  a  liaison,  and  then  a  joint  center  (such  as  the  Joint  Terrorism  Task  Force,  which 
hosts  FBI,  NCIS,  local  law  enforcement,  and  a  variety  of  Federal  agencies)  and  an  information 
system  that  automates  information  transfer  and  governs  it  through  business  rules.  We  argue  that 
organizations  with  high  adaptive  competence  choose  the  best  structure  for  the  mission,  and 
interviews  in  HI08  indicate  that  some  organizations  do  systematically  manage  the  evolution  of 
structural  forms,  typically  developing  personal  relationships  into  formal  structures.  The  literature 
on  how  and  when  to  evolve  relationships  is  scant,  however. 


Figure  4:  An  example  of  structural  evolution  in  response  to  operational  need. 
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What  are  the  competencies  that  enable  organizations  to  adapt  well  to  the  environment?  The 
competencies  must  include  the  following: 

•  Detection:  A  capability  to  detect  and  analyze  the  need  for  change,  whether  imposed  top- 
down  in  the  form  of  a  new  objective  or  mission,  or  arising  from  the  bottom  up  in  the  form 
of  unexpected  opportunities  to  accomplish  missions. 

•  Selection:  A  capability  to  select  from  experience  or  generate  organizational  structures 
and  control  mechanisms. 

•  Evaluation:  A  capability  to  think  critically  about  the  first-  and  higher-order  effects  of 
employing  these  structures  and  controls  on  the  organization,  its  partner,  other 
stakeholders,  and  the  environment. 

Recent  research  demonstrates  that  these  characteristics  enable  organizations  to  adapt  at  least  their 
procedures  to  successfully  address  novel  or  unexpected  problems.  Burke,  et  al.  (1999)  found  that 
teams  adapted  more  successfully  when  their  leaders  provided  teams  with  situational  awareness, 
made  adaptation  triggers  explicit,  and  provided  a  rationale  for  adaptation.  This  is  evidence  of  the 
need  for  a  detection  capability.  Horn  and  Zaccaro  (under  review)  provided  evidence  that  team 
leaders  (and  presumably  their  teams)  adapted  more  successfully  if  they  had  more  varied 
experience  on  which  to  draw,  evidence  of  a  selection  capability.  Finally,  Marks,  Zaccaro,  and 
Mathieu  (2000)  found  that  teams  adapted  better  when  they  were  trained  to  critique  solutions 
from  multiple  viewpoints,  evidence  of  the  need  for  an  evaluation  capability. 

We  speculate  that  adaptive  organizations  will  show  greater  evidence  of  these  competencies  than 
will  non-adaptive  organizations,  and  that  selecting  leaders  or  training  members  in  these 
competencies  may  increase  adaptivity.  Such  training  and  development  might  help  organizations 
to  ask  and  answer  the  questions  as: 

•  Detection:  Is  there  a  collaboration  or  partnership  that  will  dramatically  increase  our 
likelihood  of  success  in  this  mission?  What  does  the  partner  bring  to  this  mission?  What 
do  I  bring  to  this  mission? 

•  Selection:  How  can  we  structure  and  manage  collaboration  so  that  we  apply  resources 
efficiently,  synchronize  action,  maintain  SA,  and  exploit  opportunities  to  adapt  further? 

•  Evaluation:  How  will  the  proposed  structure  and  control  system  influence  other  aspects 
of  this  organization?  How  will  other  stakeholders  respond  to  this  partnership  and  its 
actions?  Are  collaborations  and  partnerships  that  are  forged  sustainable?  Are  they 
flexible  should  the  environment  change? 

Adaptive  competency  may  be  applied  in  long  term  planning  exercises,  and  this  may  help  ensure 
smooth,  relatively  rapid,  and  successful  development  of  partnerships.  When  it  must  be  applied  to 
emergent,  high  tempo  missions,  however,  adaptive  competency  may  be  insufficient.  Some 
organizational  infrastructure  and  control  mechanisms  may  also  be  necessary  to  adapt  at  speed. 
Assume  that  a  new  liaison  relationship  is  required  immediately  for  an  emergent  mission.  The 
organization  will  implement  this  relationship  better  and  faster  if  it  has  facilities,  information 
system  accounts,  and  training  for  incoming  liaisons  at  the  ready  at  all  times. 

In  sum,  this  draft  theory  of  HI  focuses  our  attention  and  effort  on  three  elements:  organizational 
structure,  control  mechanisms,  and  adaptive  competency.  The  challenge  ahead  of  us  is  to  define 
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experimental  eonditions  that  reveal  and  exereise  these  elements,  to  speeify  measures  with  whieh 
to  eompare  the  state  of  elements  and  their  impaet  on  mission  outeomes,  to  design  methods  that 
enable  organizations  to  aequire  the  eompeteney  to  adapt,  and  to  develop  teehniques  that  enable 
them  assess  tradeoffs  and  foreeast  effects  of  adaptation. 

Research:  HI08 

HI08  studied  the  interactions  between  organizations  conducting  Maritime  Domain  Awareness 
Scenarios  within  TWOS.  Data  were  collected  in  interviews  in  the  first  week  of  HI08  (HI08.1), 
with  law  enforcement  and  military  intelligence.  Data  were  collected  in  interviews  during  the 
second  week  of  HI08  (HI08.2)  from  DoD  organizations  tasked  with  defense  responsibilities. 

Below,  we  describe  the  objectives,  participants,  procedures,  findings,  and  recommendations  from 
this,  initial  experiment. 

Objectives 

The  overarching  objective  of  HI08  was  to  identify  enablers  of  and  constraints  on  Human 
Interoperability.  This  entailed  identifying  and  examining  interactions  at  several  levels:  human- 
human  (communication  and  behavior),  human-system,  human-organization,  and  organization- 
organization.  More  specifically,  data  collection  focused  on  several  aspects  of  organizations  that 
we  hypothesized  would  enable  or  constrain  human  interoperability: 

1 .  Intra-  and  inter-organizational  structures  that  support  collaboration,  cooperation  (sharing 
of  behaviors)  and  coordination; 

2.  Processes  that  support  collaboration,  cooperation  and  coordination; 

3.  Social,  cultural,  and  cognitive  factors  that  influence  inter-organizational  collaboration, 
cooperation  (especially)  and  coordination  by  the  various  organizations; 

4.  Policies/doctrine  concerning  information  sharing  &  cooperation  and  coordination  of 
activities;  and 

5.  Attributes  of  information  systems  that  enable  or  encumber  collaboration,  cooperation  or 
coordination. 

This  research  effort  did  not  investigate  the  role  of  individual  competency  in  organizational 
adaptation,  nor  did  it  examine  environmental  constraints  on  and  affordances  for  adaptation. 

(Both  are  components  of  the  framework  defined  above).  Information  systems  themselves  were 
not  a  focus  of  HI08,  but  we  have  captured,  below,  some  general  concerns  about  the  usability, 
utility,  socialization,  and  policies  surrounding  new  technologies. 

Participants 

The  interviewees  for  data  collection  in  HI08.1  were  drawn  from  the  Naval  Criminal  Investigative 
Service  (NCIS),  the  Joint  Terrorism  Task  Force,  the  British  Royal  Navy,  and  the  New  Zealand 
Navy.  In  addition,  data  were  collected  in  an  observation  of  a  video  teleconference  (VTC) 
between  these  organizations  and  the  Multiple  Threats  Alert  Center.  These  individuals  were 
concurrently  participating  in  the  Trident  Warrior  exercise  to  test  technologies  for  supporting 
collaboration  (only)  between  Naval  Criminal  Investigative  Service  (NCIS)  field  offices.  Joint 
Terrorism  Task  Force  (JTTF),  and  Naval  intelligence. 

The  interviewees  in  HI08.2  were  from  Third  Fleet  Joint  Forces  Maritime  Component 
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Command  (JFMCC),  the  Northern  Command  (NORTHCOM),  the  Pacific  Fleet  Maritime 
Operations  Center  (PACFLEET  MOC),  Naval  Cooperation  and  Guidance  for  Shipping 
(NCAGS),  and  the  Joint  Interagency  Task  Eorce  West  (JIATE  West).  These  individuals  were 
concurrently  participating  in  the  Trident  Warrior  experiment  to  test  a  variety  of  technologies"^ 
(see  Appendix  B;  Trident  Warrior  Technologies  in  HI08.1). 

Data  collection  was  directed  by  Stefania  Brown-VanHoozer,  Ph.D.  (OSD  Nil)  and  Shelley 
Gallup,  Ph.D.,  NPS.  Data  collection  was  conducted  by  Jared  Ereeman,  Ph.D.,  Aptima;  Randall 
Murch,  Ph.D.,  Virginia  Tech;  Tami  Smith,  from  the  office  of  the  Deputy  Undersecretary  of  the 
Navy;  and  Hal  Moore,  NORTHCOM.  Art  Glynn,  NORTHCOM  and  Jeff  Chilton,  DHS, 
contributed  to  the  data  collection  plans.  Interviewees  were  obtained  by  the  Trident  Warrior  lead 
for  Maritime  Domain  Awareness  (Dan  Dunaway,  Office  of  the  Deputy  Undersecretary  of  the 
Navy). 


Procedure 

Most  interviews  were  conducted  by  two  or  more  members  of  the  HI  research  team  with  one 
interviewee.  In  some  cases,  small  group  interviews  were  conducted.  The  interviews  were 
informal,  but  in  most  cases  addressed  all  of  the  barrier  issues  identified  above.  Representative 
questions  from  these  interviews  were: 

1 .  Tell  us  about  a  case  in  which  your  organization  encountered  challenges  collaborating 
with  another  organization  to  accomplish  your  mission. 

2.  What  structure  -  liaisons,  working  groups,  or  other  entities  -  enabled  your  organization 
to  collaborate  with  other  organizations? 

3 .  How  (by  what  processes)  did  you  execute  your  work? 

4.  What  made  this  collaboration  challenging? 

5.  Were  there  differences  in  the  culture  of  these  organizations  that  you  had  to  address? 

6.  Were  there  policies  that  enabled  or  prevented  you  from  collaborating? 

Findings 

bindings  from  HI08  interviews  are  organized,  below,  into  observations  concerning 

organizational  structure,  process,  culture,  policy,  and  human-system/program  interactions. 

Recommendations  related  to  these  findings  are  presented  in  a  subsequent  chapter  of  this  report. 

Structures  for  collaboration  between  organizations 

Interviews  conducted  in  HI08  investigated  the  range  of  structures  that  organizations  use  to 
facilitate  collaboration,  coordination  and  cooperation;  and  the  benefits  of  these  structures.  The 
informants  represented  NCIS,  the  British  Royal  Navy  Elect  Intelligence,  the  New  Zealand  Navy 
liaison  to  the  National  Maritime  Coordination  Center,  the  Maritime  Threat  Analysis  Center 
(MTAC),  and  non-governmental  organizations. 


^  These  TWOS  teehnologies  spanned  these  domains:  eommand  and  eontrol,  eoalition,  distanee  support,  fires,  human 
systems  integration,  information  operations,  ISR  (intelligenee,  surveillanee,  and  reeonnaissanee),  knowledge 
management,  networking.  Sea  Shield,  wireless  information  transport,  and  eross  domain  solutions. 
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1)  Organizational  structures  vary;  their  relationship  to  collaboration  requirements  needs 
to  he  determined 

A  variety  of  individual  and  organizational  structures  support  collaboration  and  cooperation 
between  organizations.  Examples  were  found  of  personal  trust  relationships,  liaisons,  joint  task 
forces,  third  party  intermediaries,  and  information  systems  that  pass  data  using  mutually 
agreeable  business  rules.  The  structures  have  different  strengths,  according  to  the  interviewees: 
an  NCIS  representative  to  the  JTTF,  the  British  Royal  Navy  Fleet  Intelligence,  a  New  Zealand 
Navy  liaison  to  the  National  Maritime  Coordination  Center,  members  of  the  Maritime  Threat 
Analysis  Center  (MTAC),  and  a  specialist  in  non-governmental  organizations.  More  research  is 
required  to  fully  map  out  the  utility  that  organizations  perceive  and  receive  from  different 
organizational  forms  in  various  mission  contexts. 

2)  Collaboration  and  cooperation  between  law  enforcement  organizations  depends  heavily 
on  personal  relationships  with  trusted  partners. 

The  centrality  and  sensitivity  of  personal  trust  relationships  to  law  enforcement  was  emphasized 
in  most  interviews.  These  relationships  are  built  informally  through  repeated  exchanges  of 
information  that  may  deliberately  hide  or  credit  the  source  organization,  as  the  case  requires. 
Information  exchange  can  be  conducted  without  a  strong  foundation  of  trust;  however  distrust 
can  rapidly  build  barriers  that  make  exchange  of  information  much  more  difficult.  As  noted 
below,  security  procedures  substituted  for  personal  trust  between  military  organizations  in 
exchange  of  information. 

3)  Co-location  is  a  strong  factor  in  effective  collaboration  and  cooperation. 

Though  communication  technologies  make  it  relatively  simple  to  create  distributed  teams  that 
are  physically  located  within  their  organizations,  co-location  is  widely  used.  Several  examples 
were  found  of  creating  liaisons  within  partner  organizations,  and  of  gathering  representatives  to 
create  a  new  entity  such  as  a  joint  task  force  or  working  group.  These  strategies  may  enable 
collaboration  to  occur  more  opportunistically,  ensure  coordination,  and  simplify  access  to  the 
information  and  resources  of  the  participating  organizations.  For  example,  in  the  JTTF,  the  local 
contacts  of  the  police  department  complement  the  information  resources  and  technical 
capabilities  of  Federal  agencies.  NCIS,  the  New  Zealand  Navy,  and  many  other  institutions  use 
this  strategy  when  they  co-locate  liaisons  at  the  operation  sites  of  key  stakeholders  and  provide 
civilian  entities  with  the  opportunity  to  task  military  ISR  assets.  The  JTTF  in  Hawaii  uses  this 
strategy  by  co-locating  representatives  of  the  FBI,  NCIS,  CIA,  NSA,  several  Hawaii  police 
departments,  the  FAA  Air  Marshals,  Immigration  and  Customs  Enforcement,  Coast  Guard,  the 
U.S.  Marshal’s  Service,  Secret  Service,  and  other  entities.  This  “inner  circle”  of  agencies 
expands  to  routinely  include  the  U.S.  Attorney,  Army,  National  Guard,  and  others  as  needed. 

The  number  of  participants  provides  backup  should  any  one  entity  be  unable  to  execute  a 
mission,  though  presents  a  cost  in  administrative  overhead.  JIATF  West,  similarly  co-locates 
U.S.  representatives  in  operating  centers  of  international  partners  to  provide  the  training  and 
guidance  that  may  dramatically  increase  partner  effectiveness,  accomplish  U.S.  goals  such  as 
controlling  drug  traffic  in  the  countries  of  origin  (not  just  on  American  soil),  and  build  strong 
relationships  for  future  missions.  JIATF-W  is  currently  transitioning  from  tactical  to  operational 
missions  that  require  deeper  understanding  of  policies  and  doctrine  for  international  interaction. 
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One  of  many  possible  eoneerns  about  ereating  liaisons  is  that  these  assignments  ean  hamper 
eareer  progression,  a  disineentive  to  those  who  fill  these  posts. 

4)  Collaborating  through  a  third  organization  anonymizes  exchanges  of  sensitive 
information. 

Collaborating  through  a  third  party  potentially  enables  ageneies  to  share  sensitive  information 
while  eoneealing  sources  or  methods.  The  United  Kingdom  employs  this  strategy  to  pass 
information  between  military  intelligence  and  the  police  by  interposing  the  National  Ports 
Analysis  center  between  entities  in  the  UK,  the  Mediterranean,  Balkans  and  Horn  of  Africa.  This 
entity  currently  relays  information  pertinent  to  criminal  investigations  between  intelligence  and 
law  enforcement,  and  in  the  future  may  relay  information  pertinent  to  organized  crime  and 
terrorist  activity.  Intermediary  organizations  can  be  stood  up  temporarily,  as  is  the  case  for  the 
Crisis  Action  Center  (CAC)  of  MTAC.  The  CAC  stands  up  to  manage  information  flow  from 
NCIS,  with  the  concurrence  of  the  MTAC  director  in  response  to  MOTR  calls  and  in  other  cases 
of  emergency.  This  type  of  collaboration  provides  a  means  to  limit  and/or  deny  cooperation. 

5)  Current  information  systems  support  efficient,  though  inflexible,  distributed 
collaboration. 

Information  systems  potentially  enable  participants  in  different  locations,  with  limited  personnel 
resources,  to  interoperate  through  embedded  collaboration  technologies.  NGOs  use  this  strategy 
in  humanitarian  crises.  In  response  to  the  hurricane  in  Burma,  NGOs  organized  into  functional 
clusters^  and  shared  information  in  part  through  websites.  However,  information  systems  built  to 
benefit  the  enterprise  sometimes  do  not  have  value  to  local  units,  and  so  those  technologies  are 
neither  welcomed  nor  used. 

6)  Structures  that  link  partners  consist  of  substructures  that  are  sometimes  antagonistic 
or  incongruent. 

A  given  organization  may  develop  a  complex  network  of  organizational  relationships,  some 
multilateral  (involving  information  flow  among  a  set  of  partners),  some  bilateral  (involving 
information  flow  that  is  exclusive  to  a  pair  of  organizations).  Two  competing  or  conflicting 
organizations  (e.g.,  two  law  enforcement  agencies)  may  both  fall  within  such  a  network,  and  in 
this  case  the  constraints  on  sharing  information  must  be  defined  explicitly  so  that  the 
organizations  are  confident  that  their  information  and  interests  are  secure. 

These  findings  bear  on  the  variety  of  HI  structures  to  investigate  in  future  experiments. 

Processes  for  collaboration  between  organizations 

Interviews  conducted  during  HI08.1  elicited  insights  concerning  the  processes  agencies  use  to 
facilitate  inter-agency  collaborations.  The  interviewees  represented  NCIS  at  JTTF,  the  New 
Zealand  Navy  position  within  that  nation’s  National  Maritime  Coordination  Center,  British 
Royal  Navy  Fleet  Intelligence,  Naval  Cooperation  and  Guidance  for  Shipping  (a  Navy 
organization  that  provides  advice  for  safe  passage  of  merchant  ships  worldwide),  JIATF  West, 


^  NGOs  typically  avoid  engaging  with  U.S.  government  or  military  in  crises  in  countries  opposed  to  the  U.S.  Such 
engagement  would  make  NGO  personnel  targets  of  attack  for  their  alliance  with  the  U.S. 
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and  Third  Fleet  JFMCC.  Both  informants  have  experienee  faeilitating  eollaboration  between 
military  and  law  enforeement  entities. 


7)  Effective  inter-agency  relations  are  built  on  exchanges  of  valued  assets. 

A  key  step  in  establishing  new  relationships  is  to  identify  assets  that  potential  partners  value  and 
that  the  ean  (by  poliey  or  law)  trade.  For  example: 

•  The  first  step  that  JIATF  West  takes  in  developing  new  partnerships  with  law 
enforeement  entities  in  Southeast  Asia  is  to  eontaet  the  eountry  team,  espeeially 
representatives  in  eountry,  to  learn  what  US  information  or  support  has  value  to  the 
prospeetive  partner:  training,  transportation,  information  resourees,  ete.  With  that 
knowledge,  JIATF  ean  assess  U.S.  eapability  to  provide  that  information  or  support,  and 
assign  a  program  manager  to  pool  resourees  aeross  U.S.  organizations  and  kiek  off  a 
relationship  through  exereises  and  provision  of  support  serviees. 

•  Loeal  entities  -  sueh  as  airport  authorities  and  port  authorities  -  provide  valued 
information  eoneeming  their  infrastructure,  processes,  activities,  assets,  and 
vulnerabilities  to  Federal  law  enforcement  authorities  in  exchange  for  information, 
services  (e.g.,  accelerating  a  procedure),  or  materiel  (e.g.,  access  to  technology). 

•  In  the  international  arena,  the  New  Zealand  Navy’s  military  liaison  to  the  New  Zealand 
National  Maritime  Coordination  Center  provides  the  civilian  members  with  training  to 
improve  human  resources,  and  it  provides  up  to  450  hours  of  military  ISR  support.  The 
New  Zealand  Navy,  in  return,  receives  the  support  of  specialized  personnel  for  boardings, 
certain  civilian  training,  and  access  to  an  open  source  COP  that  is  in  some  ways  superior 
to  their  classified  COP.  At  a  political  level,  the  involvement  of  the  New  Zealand  Navy  in 
the  growing  civilian  missions  of  the  NMCC  gives  the  Navy  a  rationale  to  request 
increases  in  its  budget  in  support  of  commercial  security. 

•  Third  Fleet  JFMCC  has  built  working  relations  with  a  large  number  of  organizations.  At 
the  heart  of  each  relationship  is  an  exchange  of  valued  information.  For  example,  JFMCC 
helps  Pacific  island  nations  maintain  a  shared  watch  list  of  vessels;  JFMCC  trades 
information  about  the  presence  of  vessels  from  superpowers  with  TRANSCOM;  Customs 
and  Border  Patrol  provides  cargo  data  to  JFMCC,  which  in  return  recommends  vessels 
for  boarding. 

8)  Information  exchange  costs  can  encumber  collaboration. 

A  variety  of  other  factors  can  hinder  information  flow  between  groups,  including  cumbersome 
processes  and  physical  barriers.  Processes  designed  to  protect  information  -  information 
declassification  and  requirements  to  move  information  through  trusted  sources  -  increase  the 
cost  of  doing  business,  lower  the  efficiency  of  collaboration,  and  can  decrease  trust  between 
organizations.  Some  mechanisms  do  exist  to  address  this  problem,  including  the  Intelligence 
Information  Sharing  Dispute  Resolution  (to  be  issued  as  Intelligence  Community  Policy 
Memorandum  501-3),  and  Executive  Order  12333.  Physical  barriers  can  have  similar  effects,  as 
we  heard  in  complaints  about  the  recent  sealing  of  a  door  between  PACFLEET  MOC  and  the 
JIOCC.  The  value  of  such  security  measures  should  be  carefully  weighed  against  the  costs. 

9)  Intra  and/or  Inter-organizational  partnerships  can  grow  from  opportunity  or  mandate 

(bottom-up  or  top-down). 


13 


Organizations  use  a  range  of  methods  to  discover  or  develop  potential  partnerships.  Some 
approaches  are  driven  from  the  bottom  up,  by  opportunity.  For  example,  Third  Fleet  JFMCC 
recruits  partners  in  part  through  personal  contacts  made  at  exercises,  at  conferences,  cross¬ 
briefings,  in  working  groups  and  in  other  venues.  These  partnerships  then  develop  as  a  function 
of  mutual  benefit  or  need.  Organizations  may  partner  by  mandate,  or  top  down.  For  example, 
JTTF  was  formed  through  a  strategic  national  initiative  to  combat  terrorism. 

10)  Organizations  institutionalize  personal  trust  relationships. 

By  transforming  productive,  personal  relationships  into  institutional  alliances,  organizations 
ensure  that  intra  and  inter-organizational  ties  survive  the  inevitable  transfers  and  retirement  of 
individual  relationship  holders.  For  example: 

•  An  FBI  field  office  has  sustained  its  relationship  to  municipal  police  by  inviting  them  to 
quarterly  firearms  re-certifications  (target  practice),  followed  by  a  cookout.  Police 
departments  were  invited  to  take  the  spare  ammunition  for  their  own  training  events.  This 
recurring  interaction  between  officers  helps  to  build  and  sustain  personal  trust 
relationships  that  are  important  in  solving  criminal  cases. 

•  A  British  intelligence  officer  codifies  his  personal  relationships  between  civilian  entities 
and  his  military  organization  by  (1)  identifying  a  trusted  organization  that  can  serve  as  an 
intermediary  between  information  sources  and  information  consumers,  and  (2) 
developing  procedures  that  enable  the  intermediary  to  conceal  the  information  source  and 
the  originator’s  methods,  and  transfer  it  in  a  reliable,  timely,  and  secure  manner  to  the 
consumer. 

11)  Organizations  seek  to  learn  the  capabilities  and  constraints  of  others,  and  educate 
others  about  their  own  capabilities. 

Organizations  extend  their  knowledge  of  the  capabilities  of  others,  and  may  learn  from  them,  by 
participating  in  exercises  and  by  participating  in  the  training  at  other  organizations.  NCAGS,  for 
example,  has  learned  some  exemplary  watch  processes  from  Canadians  in  exercises,  and 
participates  in  British  training  concerning  managing  and  monitoring  commercial  shipping.  The 
JTTF,  seeks  to  convey  its  joint  capabilities  to  new  staff  during  increasingly  structured  and 
comprehensive  in-briefs.  JTTF  also  involves  a  variety  of  its  members  in  operations  specifically 
to  expose  them  to  each  others  capabilities. 

12)  Organizations  require  procedures  for  triggering  and  managing  collaboration. 

Procedures  to  facilitate  collaboration  are  not  always  in  place,  or  at  least  not  in  play.  This  was  the 
case  in  parts  of  in  HI08. 1 .  For  example,  participants  at  NCIS  had  some  difficulty  determining 
when  to  reach  out  to  (other)  field  offices.  Participants  had  no  apparent  process  for  defining  a  lead 
entity,  a  “command  and  control”  node  for  an  investigation  that  crossed  venues.  Second,  a  video 
teleconference  between  MTAC  (in  DC)  and  representatives  (in  Hawaii)  of  NCIS,  Third  Fleet 
MOC,  and  the  New  Zealand  Navy  was  complex  or  confused  on  the  topics  of  how  and  with 
whom  to  collaborate  and  coordinate  across  organizations,  and  what  assets  were  available  to 
accomplish  the  mission.  The  VTC  participants  did  not  reference  any  procedures  for  making  these 
decisions  more  crisply  and  systematically.  Explicit  procedures  for  planning  and  coordinating 
joint  operations  are  often  needed  to  accomplish  missions. 
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13)  Venues  are  needed  for  testing  work  flow  processes  between  military  and  non-military 
entities. 

The  Maritime  Domain  Awareness  mission  requires  interaetion  between  military,  law 
enforeement,  and  the  vast  international  commercial  shipping  enterprise.  However,  current 
exercises  focus  largely  on  military  collaborations  within  national  military  forces  (e.g..  Trident 
Warrior)  or  between  national  military  forces  (e.g.,  Rim  of  the  Pacific  Exercise,  RIMPAC).  These 
designs  largely  ignore  or  minimize  the  role  of  commercial  shipping.  In  interviews,  NCAGS 
leadership  recommended  exercises  that  engage  commercial  shipping  at  a  realistic  scale,  and  that 
address  questions  such  as  the  following: 

•  How  can  we  pick  a  specific  Vessel  of  Interest  from  other  commercial  or  private  vessels 
almost  identical  to  it? 

•  How  do  we  communicate  securely  and  confidentially  with  a  single  ship  among  tens  or 
hundreds  in  an  area? 

•  How  can  we  coordinate  actions  with  a  crowded  field  of  internationally  flagged  white 
vessels  in  order  to  execute  mission  tasks,  given  the  complexity  of  communications  and 
the  cost  (to  commercial  interests)  of  compliance^? 

•  How  will  the  military  address  inevitable  errors  in  its  interactions  with  commercial 
carriers  and  white  vessels,  errors  that  may  lead  to  lawsuits,  future  non-compliance, 
strained  international  relations,  etc.? 

These  findings  have  implications  (presented  below)  concerning  how  to  investigate  processes  in 
HI09. 

Culture  in  inter-organizational  collaborations 

Interviews  conducted  during  HI08.1  elicited  insights  concerning  the  role  of  culture  in  inter¬ 
agency  collaborations.  The  interviewees  were  an  NCIS  special  agent  at  JTTF,  a  liaison  from  the 
New  Zealand  Navy  to  an  information  center  concerned  with  commercial  shipping,  members  of 
JIATFF  West,  and  a  member  of  the  British  Royal  Navy  Fleet  Intelligence.  Both  informants  have 
experience  facilitating  collaboration  between  military  and  law  enforcement  entities. 

14)  Historical  enmity  between  groups  can  foul  interoperability 

Deeply  ingrained  prejudices  between  groups  can  complicate  coordination,  as  one  interviewee 
illustrated  by  describing  conflicts  between  Greek  and  Turkish  entities  in  the  Sea  Breeze  2007 
exercise.  Greece  and  Turkey  have  a  history  of  conflict  stretching  back  to  the  entry  of  Turkish 
nomads  to  Greece  in  the  11**'  century,  the  subsequent  dominancy  of  Ottomans  over  Greek,  and 
the  current  stalemate  over  control  of  Cyprus. 

15)  National  cultures  differ  regarding  the  importance  of  rank. 

Members  of  JIATF-West  noted  cultural  nuances  such  as  the  importance  of  rank  may  be  more 
significant  in  Asia  than  the  U.S.  In  discussions  with  potential  Asian  collaborators,  JIATF-W 
found  it  is  often  important  to  ensure  that  the  participants  are  peers  in  rank.  This  is  consistent  with 
the  findings  of  Hofstede  (1980).  In  some  cases,  this  is  less  important  in  interactions  between  U.S. 


®  NORTHCOM  estimates  that  the  cost  of  delay  to  commercial  vessels  is  $10,000  per  hour. 
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organizations,  where  depth  of  teehnieal  knowledge  or  deeision  authority  may  be  the  essential 
elements. 

These  findings  have  implieations  eoneerning  the  aspeets  of  organizational  eulture  to  investigate 
in  HI09. 

Policy  issues  in  inter-organizational  collaboration 

Interviews  eondueted  with  representatives  of  JIATF  West,  NCIS,  and  NORTHCOM  produeed 
useful  insights  eoneerning  the  role  of  poliey  in  inter-organization  eollaborations. 

16)  Policies  affecting  information  flows  between  organizations  or  across  secure  domains 
differ,  and  often  conflict,  between  military  and  law  enforcement  entities. 

•  Military  organizations  have  formal  polieies  governing  the  distribution  of  sensitive 
information.  This  formal  proeess  obviates  the  need  for  personal  trust  relationships  (1)  to 
move  information  quiekly  between  organizational  elements  that  do  not  share  personal 
relationships,  (2)  when  the  trustworthiness  of  members  is  established  a  priori  through 
seeurity  elearanees,  and  (3)  when  personnel  rotate  frequently.  However,  it  eneumbers 
eollaboration,  and  thus,  eooperation,  with  some  partners,  ineluding  foreign  military  and 
domestie  law  enforeement  (below). 

•  Law  enforeement  entities  often  pass  sensitive  information  primarily  on  the  basis  of  trust, 
through  interpersonal  relationships  with  known  individuals.  The  relianee  on  trust  obviates 
the  need  for  development  of  eertain  polieies  and  proeedures,  works  well  when  the 
organization  must  internet  with  other  entities  that  laek  sueh  polieies  and  proeedures,  and 
provides  flexibility  based  on  individual  judgment  about  trustworthiness  in  eontext.  We 
note,  however,  that  information  sharing  in  law  enforeement  will  inereasingly  beeome 
subjeet  to  poliey  (business  rules)  as  it  is  automated  by  information  systems  sueh  as  LInX 
(see  Appendix  B;  Trident  Warrior  Teehnologies  in  HI08.1). 

17)  Organizations  differ  on  how  proactive  they  are  about  refining  policies. 

JIATF-West  representatives  take  an  assertive  approaeh  to  resolving  eonfliets  between  mission 
requirements  and  poliey.  Speeifieally,  JIATF  West  often  pushes  its  reeommendations  for  poliey 
revisions  up  the  ehain  of  eommand.  This  strategy  for  eontrolling  polieies  from  the  bottom  up  was 
not  mentioned  by  other  organizations. 

Information  systems  for  collaboration  between  organizations 

Information  systems  designed  to  ensure  and  aeeelerate  information  flow  sometimes  present 
ehallenges  to  organizations  and  to  inter-organizational  eollaboration,  aeeording  to  interviewees 
representing  LInX,  NCIS,  the  New  Zealand  and  British  Navies,  JIATF  West,  NORTHCOM, 
JFMCC,  and  in  related  MDA  researeh  with  NAVCENT. 

18)  Information  can  be  classified  unintentionally;  the  cost  of  declassification  to  support 
collaboration  can  be  high. 
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Information  is  sometimes  elassified  simply  by  its  passage  onto  restrieted  military  networks  and 
systems,  though  it  may  not  warrant  olassifieation’  (per  the  eategories  speeified  in  Exeeutive 
Order  12958).  The  proeess  of  deelassifying  those  data  is  manual,  slow,  and  thus,  rarely  used. 

This  makes  it  diffieult  to  transfer  important  information  from  the  military  to  law  enforcement 
organizations,  to  coalition  partners,  and  to  others. 

19)  Delivery  of  systems  that  local  users  cannot  support  (“drive  by  fielding”)  diminishes 
return  on  technology  investments. 

As  interviewees  at  JIATF-West  indicated  information  systems  designed  to  serve  the  enterprise 
may  lack  local  utility,  and  systems  are  sometimes  discarded  for  that  reason.  More  specifically, 
interviewees  reported  that  technologies  are  sometimes  pushed  on  their  organizations  though  they 
do  not  have  sufficient  staff  to  use  them,  lack  training  that  clarifies  how  to  apply  the  technology  to 
their  specific  mission,  and  find  the  systems  to  be  extraordinarily  difficult  to  use. 

20)  Communities  of  interest  can  help  unify  users  and  standardize  use. 

In  the  case  of  LInX,  disparate  policies  about  information  sharing  have  developed  in  the  different 
LInX  regions.  However,  the  LInX  program  has  addressed  this  by  creating  regional  steering 
committees  that  set  policy,  and  an  executive  steering  committee  that  attempts  to  standardize 
policy  across  regions. 

21)  Information  quality  must  be  assured  for  user  confidence. 

Information  quality  can  vary  greatly  within  a  system.  Users  of  LInX,  for  example,  must  often 
call  the  originating  law  enforcement  office  to  interpret  records,  must  deconflict  coincidental 
relationships  between  records  (e.g.,  the  same  name  may  be  used  for  different  suspects),  and  must 
correlate  records  that  appear  to  differ  but  in  fact  concern  the  same  individual  (e.g.,  different 
name  spellings  or  aliases  may  be  used  for  the  same  suspect).  Cost-effective  methods  of  assuring 
information  quality  are  needed. 

22)  Information  systems  and  systematic  information  handling  create  value  and  advertise 
expertise. 

Third  Fleet  JFMCC  has  used  a  home-grown  information  system  to  establish  itself  as  an  efficient 
broker  of  critical  information  about  white  shipping  to  military  intelligence  and  operational 
forces.  Specifically,  JFMCC  has  developed  technology  that  prioritizes  vessels  for  action  by 
partners  at  DoD  and  in  law  enforcement.  The  technology  also  enables  JFMCC  to  turn  around 
requests  for  information  in  minutes,  rather  than  a  day.  Like  the  best  information  systems,  this  one 
effectively  exposes  and  advertises  the  expertise  of  JFMCC  (its  categorization  scheme  and 
business  rules)  so  that  clients  can  inspect  and  negotiate  JFMCC ’s  prioritization  judgments  about 
vessels,  calibrate  their  trust  in  JFMCC,  and  act  on  its  judgments. 

23)  Information  systems  have  hidden  dependencies  that  can  foil  collaboration. 

Participants  in  the  law  enforcement  component  of  TWOS  (HI08.1)  had  some  unexpected 
technical  problems  with  collaboration  technologies.  These  included  lack  of  familiarity  with 


^  A  NORTHCOM  representative  reports  that  70%  of  all  work  exeeuted  on  SIPRNET  is  unelassified. 
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required  file  types  (e.g.,  the  .pdf  format  for  documents)  or  software  to  open  them,  and  lack  of 
connectivity  (e.g.,  lack  of  unclassified  data  ports).  More  generally,  systems  have  dependencies 
on  user  knowledge  or  certifications,  operating  environment,  and  mission  that  often  are  not  made 
explicit,  and  thus  present  unwelcome  surprises  to  users. 

These  findings  have  implications  concerning  the  interaction  between  information  systems  and 
the  organizations  they  are  designed  to  support. 


Discussion  and  Recommendations 

In  this  section,  we  draw  implications  from  the  findings  above,  and  we  leverage  HI  theory  to 
specify  objectives  for  future  research.  We  refer  to  the  next  research  events  as  HI09,  in 
anticipation  of  the  creation  of  an  HI  research  program. 

Implications  of  HI08  research  findings 

For  each  of  the  findings  documented  above,  we  have  drawn  one  or  more  implications  for  leaders 
and  managers  of  inter-  and  intra-organizational  collaborations.  These  suggestions  are 
preliminary.  By  that,  we  mean  that  they  arise  naturally  from  the  interview  findings  of  HI08,  but 
they  should  be  validated  in  a  broader  and  deeper  program  of  research  concerning  human 
interoperability. 


Table  1:  Implications  of  HI08  research  findings 


Factor 

Finding 

Implications 

Structure 

1)  Organizational  structures 
vary;  their  relationship  to 
collaboration  requirements 
needs  to  be  determined 

Support  research  into  the  form  and  function  of 
structures  for  collaboration.  Educate  leaders 
concerning  the  effectiveness,  costs,  and  benefits 
of  these  options. 

2)  Collaboration  and 
cooperation  between  law 
enforcement  organizations 
depends  heavily  on  personal 
relationships  with  trusted 
partners. 

In  partnering  across  professional  domains  (e.g., 
military  to  law  enforcement),  give  preference  to 
forms  of  partnership  that  are  familiar  to 
prospective  collaborators  (e.g.,  personal  trust 
relationships  for  law  enforcement).  Develop  and 
train  reliable  methods  of  building  quick  trust. 
Develop  certifications  of  trustworthiness  (e.g., 
demonstrable  proof  of  competency)  to  accelerate 
development  of  trust  between  organizations  that 
have  the  potential  to  collaborate.  Some 
technology  solutions  may  be  required. 
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3)  Co-location  is  a  strong 
factor  in  effective 
collaboration  and 
cooperation. 

Carefully  weigh  the  eeonomy  of  distributed  or 
virtual  teams  against  the  value  of  personal  trust 
relationships  and  insights  gained  through  eo- 
loeation  of  personnel.  Where  liaison 
requirements  ean  be  antieipated,  arrangements 
should  begin  early  enough  to  find  the  fit  between 
the  position,  the  organization,  and  the 
environment  or  problem  being  worked.  Where 
they  ean’t  be  antieipated,  ensure  that  processes 
(e.g.,  training)  and  infrastructure  (e.g.,  facilities) 
are  in  plaee  to  create  them  rapidly. 

4)  Collaborating  through  a 
third  organization 
anonymizes  exchanges  of 
sensitive  information. 

Consider  establishing  third  party  intermediaries 
or  information  infrastrueture  to  move 
information  when  (1)  sources  and  methods  must 
be  eoneealed  (2)  and  findings  must  be  rapidly 
eommunieated  between  organizations  that  do  not 
have  interoperable  information  systems  (e.g., 
eommon  access  to  SIPRNet  or  IntelLink). 

5)  Current  information 
systems  support  effieient, 
though  inflexible,  distributed 
eollaboration. 

In  assessing  value  of  proposed  information 
systems,  evaluate  the  utility  to  the  enterprise  and 
to  eaeh  of  its  members.  Ensure  that  relatively 
inflexible  systems  (e.g.,  highly  struetured 
databases)  return  sufficient  value  to  users  given 
the  eost  of  using  them.  Make  flexibility  a  design 
eriterion  for  information  systems;  discourage 
“stovepipe”  systems. 

6)  Structures  that  link 
partners  consist  of 
substructures  that  are 
sometimes  antagonistie  or 
incongruent. 

Assess  the  eompatibility  of  partners  -  their 
history,  goals,  proeesses,  and  eulture  -  before 
investing  heavily  in  developing  partnerships. 
Invest  in  monitoring  and  managing  potential 
incompatibilities. 

7)  Effeetive  inter-ageney 
relations  are  built  on 
exehanges  of  valued  assets. 

Develop  and  exereise  eapability  to  assess  the 
value  of  assets  eaeh  party  brings  to  a 
relationship.  Make  trade  of  these  assets  an 
explieit  goal  of  partnership  agreements. 

Processes 

8)  Information  exehange 
costs  can  encumber 
collaboration. 

Assess  the  eosts  of  exehanging  information  (e.g., 
delay  or  information  loss  due  to  deelassifieation) 
and  other  assets.  Manage  these  costs  and  partner 
perceptions  of  them. 

9)  Intra  and/or  Inter- 
organizational  partnerships 
ean  grow  from  opportunity  or 
mandate  (bottom-up  or  top- 
down). 

Eormalize  methods  of  identifying  and  growing 
partnerships  from  the  bottom  up,  and  the  top 
down.  Ensure  that  these  methods  are  in  place 
and  function  for  time  critical  and  unanticipated 
eonfigurations. 
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10)  Organizations 
institutionalize  personal  trust 
relationships. 

Manage  personal  trust  relationships  as  an 
enterprise  asset  by  identifying  them  and 
developing  them  selectively  into  enduring 
structural  entities.  Define  working  styles  and 
processes  that  enable  lower  the  amount  of 
interpersonal  interaction  required  to  build  or 
verify  trust  in  emergent  organization 
relationships. 

11)  Organizations  seek  to 
learn  the  eapabilities  and 
constraints  of  others,  and 
educate  others  about  their 
own  capabilities. 

Develop  systematic  methods  of  evaluating 
potential  partners,  and  of  representing  your  own 
organization’s  capabilities  to  support  others  in 
unique  or  time  critical  situations. 

12)  Organizations  require 
procedures  for  triggering  and 
managing  collaboration. 

When  planning  missions  of  significant 
complexity  and  missions  in  new  domains, 
systematically  assess  the  utility  and 
manageability  of  collaboration.  In  preparation 
for  emergent,  unforeseen  missions,  develop 
mechanisms  that  accelerate  this  assessment  or 
obviate  the  need  for  it. 

Culture 

14)  Historical  enmity 
between  groups  can  foul 
interoperability 

Identify  areas  of  historic  conflict  between 
potential  partners.  Monitor  and  manage 
problems  that  arise  from  these  seemingly 
irrelevant  sources. 

15)  National  cultures  differ 
regarding  the  importance  of 
rank. 

When  staffing  meetings  that  cross  cultures, 
attend  to  partners’  requirements  for 
showing/seeing  authority  and  technical 
expertise. 

Policy 

16)  Policies  affecting 
information  flows  between 
organizations  or  across 
secure  domains  differ,  and 
often  conflict,  between 
military  and  law  enforcement 
entities. 

Ensure  that  partners  make  information  sharing 
constraints  and  mechanisms  explicit. 

17)  Organizations  differ  on 
how  proactive  they  are  about 
refining  policies. 

Define  channels  and  methods  for  pushing  critical 
policy  revisions  upward. 

Information 

systems 

1 8)  Information  can  be 
classified  unintentionally;  the 
cost  of  declassification  to 
support  collaboration  can  be 
high. 

Identify  information  critical  to  collaboration  that 
may  be  unintentionally  compartmentalized. 

Ensure  alternative  access  routes  to  this 
information. 

20 


19)  Delivery  of  systems  that 
local  users  cannot  support 
(“drive  by  fielding”) 
diminishes  return  on 
technology  investments. 

Provide  transitional  support  and  utility 
assessment  with  the  delivery  of  new 
technologies. 

20)  Communities  of  interest 
can  help  unify  users  and 
standardize  use. 

Establish  communities  of  interest  around 
technologies  and  practices  that  deliver  high 
value  information. 

21)  Information  quality  must 
be  assured  for  user 
confidence. 

Ensure  that  partnering  organizations  understand 
which  information  elements  are  least  reliable. 
Develop  procedures  for  testing  and  verifying  this 
information. 

22)  Information  systems  and 
systematic  information 
handling  create  value  and 
advertise  expertise. 

Proceduralize  information  handling  and  structure 
information  products  where  it  is  feasible, 
productive,  and  affordable  to  do  so.  Advertise 
and  share  those  products,  invite  critiques  to 
increase  product  value. 

23)  Information  systems  have 
hidden  dependencies  that  can 
foil  collaboration. 

Ensure  that  users  and  their  systems  satisfy  the 
requirements  for  using  new  systems  on  critical 
tasks. 

Objectives  for  HI09  research 

The  goal  of  the  Human  Interoperability  initiative  is  to  develop  knowledge  and  methods  that 
improve  the  responsiveness,  efficiency,  and  effectiveness  of  organizations  that  partner  or 
dynamically  integrate  their  own  units.  The  implications  of  HI08,  reported  above,  provide  some 
insight  into  the  guidance  that  HI  research  can  offer.  They  must,  however,  be  extended,  deepened, 
and  developed  into  a  body  of  knowledge  that  can  be  taught,  executed  by  organizational  leaders 
and  membership,  implemented  in  organizational  process  architectures,  incorporated  into  decision 
aids,  and  converted  into  measures  that  support  monitoring  of  organizational  state  and  effects. 

Much  of  what  we  have  learned  in  HI  08  has  resolved  into  several  themes: 

o  Organizations  must  build  on  historical  experiences  to  improve  human  interoperability  in 
collaborations  they  can  foresee; 

o  Organizations  must  develop  mechanisms  that  allow  them  to  partner  rapidly  and 
effectively  under  conditions  that  cannot  be  foreseen,  because  these  are  inevitable; 

o  Organizations  must  develop  the  self-awareness  to  assess  their  capability  and  performance 
in  both  of  these  possible  states. 


To  address  these  themes  requires  research  that  defines  (1)  useful  organizational  structures  and 
controls  used  in  the  past,  (2)  human  dimensions  that  enable  organizations  to  adapt  creatively  to 
unforeseen  future  states,  and  (3)  technical  means  that  may  serve  as  proxies  for  the  time-tested  but 
time-consuming  human  interactions  on  which  successful  interactions  are  historically  based. 
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We  recommend  a  program  of  research  concerning  Human  Interoperability,  one  that  explores  the 
challenges  and  solutions  to  Human  Interoperability.  It  is  fundamental  to  such  research  that  we 
specify  its  objectives  in  detail.  We  do  so,  below,  by  presenting  objectives  and  research  questions 
related  to  each  of  the  dimensions  of  the  draft  theory  (see  page  2,  “Toward  a  theory  of  Human  I”): 
organizational  structures,  adaptive  competency,  structure,  policy,  procedures,  culture,  cognitive 
competency,  and  data  interoperability. 

•  Adaptive  competencies 

o  Objective:  Identify  the  adaptive  competencies  of  organizations 
o  Research  questions: 

■  How  do  organizations  perceive  triggers  or  cues  to  change  structure, 
processes  and  control  mechanisms? 

■  How  do  organizations  select  partners? 

■  How  do  organizations  capture  and  test  knowledge  about  the  capabilities  of 
partner  organizations?  How  and  when  do  they  communicate  their  own 
capabilities? 

■  What  is  the  variety  of  experiences  that  enable  adaptation? 

■  How  do  organizations  select  or  generate  feasible  structures  and  controls 
and  under  what  conditions? 

■  How  do  organizations  anticipate  or  react  to  direct  and  indirect  effects  of 
adaptation? 

•  Structure 

o  Objectives:  Assess  structural  barriers  to  inter-organization  collaboration  and 
mechanisms  for  overcoming  them 
o  Research  questions: 

■  What  structures  do  organizations  have  in  place  that  support  collaboration 
with  other  organizations? 

■  What  structures  do  organizations  create  for  collaboration?  How  elastic  are 
organizations  to  new  structural  forms?  What  influences  this  flexibility? 

■  What  are  the  evolutionary  patterns  of  structural  change  (e.g.,  from 
personal  relationships  to  working  groups)? 

■  What  is  the  effect  on  intra-  and  inter-organization  collaboration  when 
antagonists  enter  the  membership  network? 

•  Policy 

o  Objective:  Identify  policy  barriers  to  inter-organization  collaboration  and 
mechanisms  for  overcoming  them 
o  Research  questions: 

■  Are  members  of  organizations  able  to  identify  policy  barriers,  violations 
of  policy,  and  new  policy  requirements? 

■  Is  the  organization  able  to  modify  or  work  around  policy  barriers  when 
needed? 

•  Process 

o  Objectives:  Assess  procedural  barriers  to  inter-organization  collaboration  and 
mechanisms  for  overcoming  them 
o  Research  questions: 
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■  How  do  organizations  specify  who  coordinates  with  whom,  when,  how, 
and  based  on  what  information  or  circumstances?  Are  those  procedures 
known  in  the  organization? 

■  What  is  the  valued  information  or  capability  that  is  exchanged  between 
organizations?  What  are  the  benefits  and  costs  of  exchanging  information? 

■  Do  organizations  have  mechanisms  to  identify  and  institutionalize 
personal  relationships? 

•  Culture 

o  Objectives:  Identify  barriers  at  the  cultural  interface  between  organizations 
o  Research  questions 

■  Does  the  effectiveness  of  partnerships  vary  with  differences  in  power 
distance,  individualism,  uncertainty  handling,  short-  vs.  long-term 
perspective  and  other  factors? 

■  How  do  organizations  resolve  or  manage  cultural  conflicts? 

•  Cognition 

o  Objective:  Identify  the  barriers  to  individual  ability  to  identify,  form,  and 

implement  collaborations  and  specify  the  knowledge,  skills,  and  attitudes  required 
to  overcome  these  barriers, 
o  Research  questions: 

■  What  competencies  enable  individuals  to  lead  inter-  and  intra- 
organizational  partnerships? 

■  What  competencies  enable  individuals  to  participate  fully  in  partnerships? 

•  Data  interoperability 

o  Objectives:  Assess  barriers  to  collaboration  at  the  information  interface  and 
mechanisms  for  overcoming  them 
o  Research  questions 

■  How  do  organizations  balance  and  rebalance  the  requirements  to  protect 
and  to  disseminate  sensitive  but  valued  information? 

■  Do  technology  users  have  staff,  mission-relevant  training,  resources  and  a 
community  of  interest  to  support  use  and  maintenance  of  new  technology? 

■  What  methods  of  information  quality  assurance  are  used  to  test  and 
communicate  the  validity  of  data? 

Methods  of  HI09  research 

Research  planning  specifies  measures,  materials,  venues,  and  procedures.  NFS  has  refined 

techniques  for  this  through  its  leadership  of  analysis  activities  in  all  Trident  Warrior  experiments. 

In  brief,  this  technique  involves  the  following  steps: 

o  Measures  are  defined  that  explore  and  test  the  potential  answers  to  each  research 
question.  Measurement  instruments  are  developed  for  use  in  research  studies. 

o  Venues  are  identified  in  which  research  can  be  conducted.  These  include  field  sites  at 
which  observation  and  interviews  can  be  conducted,  tabletop  exercises  in  which 
organizational  leaders  demonstrate  the  process  and  outcomes  of  decision  making, 
laboratory  experiments  in  which  alternative  tools  and  treatments  are  contrasted,  and 
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military  experiments  that  exereise  operational  personnel  and  teehnology  on 
representative  missions. 

o  Seenarios  and  scenario  threads  are  defined  for  selected  venues.  These  scenarios  specify 
opportunities  to  apply  measures. 

o  Procedures  are  defined  to  train  research  teams  (e.g.,  concerning  when  and  where  to  use 
measurement  instruments);  train  research  participants  in  technologies,  procedures, 
structures,  and  missions  they  will  use;  install  technologies  required  for  the  study; 
schedule  and  synchronize  events;  analyze  data;  and  generate  reports. 

These  activities  are  executed  synchronously  in  meetings,  and  asynchronously  using  collaboration 
technology,  such  as  the  NPS  website  called  FIRE:  FORCEnet  Innovation  &  Research  Enterprise. 
That  site  provides  collaboration  functions:  email,  voicemail,  calendar,  task  lists,  discussion 
forums,  web  conferencing,  fileshare,  and  administrative  controls.  Critically,  many  of  the 
products  of  activities  executed  live  and  through  the  workspace  are  stored  in  a  database  - 
accessible  to  the  entire  team  of  researchers  and  stakeholders  -  that  relates  objectives,  measures, 
scenarios,  venues,  and  procedures.  This  highly  structured  method  of  developing  and 
disseminating  product  significantly  improves  management  of  experiments,  and  accelerates 
analysis  and  reporting. 

A  number  of  more  detailed  ideas  concerning  future  experiments  are  presented  in  Appendix  E: 
Notes  Towards  Experimentation. 

Conclusion 

This  document  reports  a  framework  for  understanding  the  Human  Interoperability,  data 
concerning  Human  Interoperability  at  Trident  Warrior  2008,  implications  of  those  findings,  and 
objectives  for  future  research.  That  research  must  refine  HI  theory,  report  findings  from 
systematically  designed  studies  using  quantitative  measures,  and  develop  products  -  case  studies, 
guidelines,  and  diagnostic  and  predictive  models  -  that  help  organizations  grow  their  capabilities 
to  interface  at  the  human  level  with  their  current  and  future  partners. 


*  We  remind  the  reader  that  experiments  eondueted  by  DoD  differ  from  the  eontrolled,  laboratory  events  eondueted 
in  seienee  and  engineering  eommunities.  DoD  experiments  often  injeet  a  new  teehnology  or  proeess  into  an  existing 
organization  or  operational  system.  No  eontrol  eonditions  are  established.  The  effeets  of  the  manipulation  are 
generally  assessed  by  subjeet  matter  experts  and  by  partieipants  who  have  substantial  personal  experienee  with 
standard  teehnologies  or  proeesses.  Further,  DoD  experiments  serve  several  funetions  above  and  beyond  the 
diseovery  of  effeets.  These  events: 

•  Advertise  new  teehnologies  and  proeedures  to  partieipants 

•  Test  the  interoperability  of  teehnologies 

•  Test  training  for  these  teehnologies  and  proeedures 

•  Produee  nominal  plans  for  exeeuting  the  tested  mission,  as  well  as  new  taeties,  teehniques,  and  proeedures 

•  Produee  knowledge  of  how  enemy  forees  may  respond  to  ehanges  in  U.S.  teehnologies  or  proeesses 
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Appendix  A:  Literature  Review 


Several  bodies  of  literature  bear  on  the  phenomena  we  call  the  Human  Interoperability:  (1) 
theoretical  and  empirical  research  by  industrial  /  organizational  psychologists  concerning  team 
effectiveness,  (2)  design  and  testing  techniques  of  systems  engineers  (3)  and  cross-cultural 
studies.  None  of  these  literatures  satisfactorily  inform  the  analysis,  design,  development,  and 
testing  of  the  human  interface  between  organizations.  So  that  the  reader  can  draw  their  own 
conclusions  regarding  this  claim,  we  provide  an  overview  of  this  literature,  below. 

Psychology  of  Team  Effectiveness 

Industrial  /  organizational  psychologists  have  developed  a  diverse  body  of  theory  to  explain  how 
the  competencies  and  behaviors  of  team  members  influence  the  effectiveness  of  teams.  This 
work  is  characterized  by: 

•  A  focus  on  team  members,  rather  than  the  environment  in  which  they  work 

•  A  focus  on  interactions  within  teams,  rather  than  between 

•  No  assumptions  that  the  activities  of  teams  are  well-defined  or  ill-defined 

•  A  large  number  of  abstract  constructs  (e.g.,  leadership)  that  are  experimentally 
operationalized  in  a  variety  of  ways,  rather  than  a  limited  number  of  relatively  concrete 
factors 

•  Research  methods  that  apply  surveys  of  team  members  or  observers  in  operational  or 
training  settings,  rather  than  laboratory  methods 

The  flavor  of  this  literature  is  evident  in  a  few,  representative  theories  selected  a  brief  survey  of 
this  literature.  (For  a  more  extensive  review,  see  Salas,  Stagl,  and  Burke,  2004.) 

Salas,  Sims,  and  Burke  (2005)  drew  on  a  broad  body  of  theory  and  research  (Salas,  Stagl,  and 
Burke,  2004)  to  define  a  “Big  Five”  factors  in  teamwork^: 

•  Team  leadership 

•  Mutual  performance  monitoring 

•  Backup  behavior 

•  Adaptability 

•  Team  orientation,  an  attitude  favoring  interdependence  and  interaction 

They  identified  several  mechanisms  by  which  teams  coordinate  to  apply  these  five  capabilities 
well: 

•  Shared  mental  models 

•  Mutual  trust 

•  Closed-loop  communication 

Prior  work  by  McIntyre  and  Salas  (1995)  touched  on  these  factors  and  others  in  a  review  of 
survey  research  conducted  on  military  teams  that  execute  naval  gunfire  support,  antisubmarine 


^  The  phrasing  is  a  play  by  Salas,  Sims,  and  Burke  (2005)  on  the  Big  Five  personality  factors:  neuroticism, 
extraversion,  openness  (or  intellect),  agreeableness,  and  conscientiousness  (Thurstone,  1934). 
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warfare,  and  guided  missile  operations.  The  authors  specified  20  principles  that  touched  on  four 
of  the  “Big  Five”  factors  (excluding  adaptability),  as  well  as: 

•  Feedback  between  team  members 

•  Taskwork  skills  of  individual  team  members 

•  Context-driven  selection  of  skills  and  styles  (e.g.,  authoritarian  vs.  interactive  style)  from 
a  large  repertoire 

•  Evolution  of  skills  over  time 

In  a  broader  review,  Cannon-Bowers,  Tannenbaum,  Salas,  and  Volpe  (1994)  organized  what 
they  termed  the  “confused  and  contradictory”  literature  on  team  skills  into  eight  factors,  several 
of  them  familiar  from  the  lists,  above: 

•  Adaptability  of  teamwork  strategies  to  the  environment 

•  Shared  situational  awareness 

•  Performance  monitoring  and  feedback 

•  Leadership  and  team  management 

•  Interpersonal  relations  to  optimize  cooperation 

•  Coordination  of  team  activities 

•  Communication 

•  Decision  making 

There  is,  then,  some  consensus  concerning  factors  and  behaviors  between  individuals  that  enable 
teams  to  be  effective.  It  is  not  clear,  however,  what  role  these  factors  play  as  teams  and 
organizations  build  alliances  with  others. 

In  related  work,  personality  researchers  have  developed  a  small  but  growing  body  of  evidence 
that  individual  personality  characteristics  -  not  just  individual  taskwork  proficiency  (Stewart, 
2006)  -  influence  team  performance  (Van  Scotter  and  Motowidlo,  1996;  Day  and  Silverman, 
1989;  Organ  and  Ryan,  1995;  Hogan,  Rybicki,  Motowidlo,  S.,  and  Borman,  1998).  Because 
personality  characteristics  are  not  highly  malleable,  this  literature  has  more  utility  with  respect  to 
the  selection  of  team  members  than  their  training  or  management.  However,  it  is  not  entirely 
clear  how  much  variance  personality  characteristics  account  for  given  other  team-level  variables 
(above). 


Systems  Engineering  Approaches  to  Team  Design 

The  systems  engineering  community  has  developed  methods  of  representing  teams,  in  efforts  to 
design  organizations  or  the  systems  they  command  and  operate.  This  work  implicitly  defines  a 
theory  of  human  interoperability,  because  it  specifies  the  environment  in  which  people  operate. 
The  characteristics  of  this  work  in  large  part  contrast  with  those  of  team  effectiveness  research. 
In  the  design  literature,  one  finds: 

•  A  focus  on  the  environment  in  which  teams  function,  rather  than  on  the  team  members 

•  A  focus  on  phenomena  between  organizational  entities  (typically  teams),  rather  than  on 
activity  with  those  entities 

•  An  assumption  that  the  activities  of  teams  are  well-defined,  and  thus,  that  they  can  be 
formally  represented  and  modeled 
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•  A  small  number  of  relatively  eonerete  eonstruets  (entities,  resourees,  tasks)  ,  rather  than  a 
large  number  of  abstraet  faetors 

•  Research  that,  though  scant,  employs  mainly  laboratory  methods,  rather  than  survey 
techniques  in  operational  settings 

A  few  examples  represent  this  field  well. 

The  DoD  Architecture  Framework  (Department  of  Defense,  2007)  specifies  methods  of 
representing  the  operational  aspects  of  a  system  (“OV”  diagrams  of  its  function,  connectivity, 
information  dependencies,  roles  and  relationships,  activities,  and  states),  technical  aspects  of 
system  entities  (“SV”  diagrams  of  entity  interfaces,  functions,  activities,  performance 
parameters,  technologies,  information  formats,  and  others),  and  standards  employed  by  the 
system  (“TV”  diagrams). 

A  small,  more  diverse,  and  largely  academic  community  of  engineers  has  defined  other  methods 
of  representing  and  computationally  modeling  the  activity  of  organizations.  This  work  applies 
multi-objective  optimization  techniques  (Levchuk,  Chopra,  Levchuk,  and  Paley,  2005)), 
simulated  annealing  (Carley  and  Svoboda,  1996),  discrete  event  simulation  (Barnes  and 
Laughery,  1997),  network  analytic  methods  (Levchuk  and  Chopra,  2005)  and  other  techniques 
(Levitt,  2004)  to  design,  test,  or  infer  the  command  and  control  relationships  among 
organizational  entities  (their  structure)  and  the  precedence  of  tasks  (their  process).  These  models 
represent  a  variety  of  factors,  typically  some  subset  of: 

•  Mission  /  Process  -  defined  as  a  task  precedence  graph 

•  Events  -  defined  as  cues  to  task 

•  Tasks  -  characterized  by  the  resources  required  to  execute  them 

•  Assets  /  Resource  -  the  capabilities  of  materiel  entities  to  perform  tasks 

•  Role  /  Decision  maker  -  the  capabilities  of  human  entities  to  perform  tasks 

•  Organization  -  the  command,  control,  and  communication  structures  that  constrain 
authority  over  people,  asset  allocation,  and  information  exchange 


Integration  of  Psychological  and  Engineering  Approaches 

A  recent  body  of  work  combines  the  I/O  and  systems  engineering  approaches  to  understanding 
and  designing  effective  human  organizations. 

Research  by  Ilgen  and  Holloway  (Moon,  Hollenbeck,  Ilgen,  West,  Ellis,  Humphrey,  Porter, 
2000)  uses  experimental  methods  and  statistical  modeling  techniques  characteristic  of  the  social 
sciences  to  explore  the  behavior  of  organizations  that  are  structured  in  ways  that  systems 
engineers  would  readily  recognize.  Their  work  has  discovered  surprising  effects,  such  as  an 
asymmetry  in  structural  adaptation  by  which  teams  more  readily  shift  their  structure  from 
functional  (in  which  each  member  controls  a  unique  type  of  assets,  e.g.,  aircraft)  to  divisional  (in 
which  each  member  controls  all  types  of  assets  in  a  given  geographical  region)  than  from 
divisional  to  functional,  which  the  mission  conditions  demand  structural  adaptation. 

Orvis  and  colleagues  (Orvis  and  Zaccaro,  2007;  McCormack,  Ducchon,  Geyer,  and  Orvis,  in 
press)  have  addressed  to  problem  of  composing  effective  teams  by  representing  factors  from  the 
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I/O  tradition  and  the  systems  engineering  paradigm  in  a  single  model.  This  approach  is  intended 
to  specify  teams  whose  members  possess  the  competencies  to  perform  well-specified  tasks,  are 
available  to  perform  those  tasks,  and  have  the  leadership  and  followership  (team  skills) 
competencies  to  collaborate  well  with  teammates. 

Skarin  (personal  communication,  25  September  2008)  is  approaching  the  problem  of  team 
composition  by  modeling  the  knowledge  and  knowledge  requirements  of  team  members. 
Statistical  language  processing  techniques  are  used  to  infer  the  knowledge  of  members  of  an 
organization  from  the  content  of  documents  they  read  and  write.  The  relationships  among  those 
individuals  are  mapped  as  social  networks  from  data  concerning  email  correspondence,  phone 
calls,  and  physical  proximity  (using  RFID  badges).  These  data  are  sufficient  to  make  inferences 
concerning  which  individuals  currently  outside  a  team  might  bring  needed  knowledge  into  that 
team. 

Cross-Cultural  Studies 

Researchers  of  culture  vary  in  their  methods  from  the  largely  exploratory  methods  of 
anthropology,  which  are  designed  to  enable  relevant  social  constructs  to  emerge  through 
observation  and  participation,  to  more  formal  schools  that  employ  survey  methods  identify 
constructs  or  dimensions  that  reliably  distinguish  one  culture  from  another.  We  focus  here  on  the 
more  formal  school,  whose  work  is  characterized  by: 

•  A  focus  on  the  society  or  culture,  rather  than  on  individuals,  teams,  or  their  environment 

•  A  focus  on  reliable  differences  in  the  character  of  each  culture,  rather  than  on  interactions 
between  organizations  or  individuals 

•  No  representation  of  tasks,  and  thus  no  assumptions  that  they  are  well-defined  or  ill- 
defined 

•  A  modest  number  of  relatively  abstract  constructs  (e.g.,  authority  relationships)  ,  rather 
than  a  large  number  of  abstract  factors 

•  Research  using  surveys  whose  items  often  are  not  grounded  (or  constrained  by)  specific 
context  or  missions,  and  thus  may  lack  validity. 

The  current  touchstone  in  this  field  is  Hofstede’s  (2004)  five  dimensional  cultural  model,  in 
which  cultures  are  characterized  in  terms  of  power  distance,  which  reflects  the  degree  of  equality 
or  inequality  among  the  population;  individualism  V5.  collectivism,  the  degree  to  which 
individual  or  collective  achievements  are  rewarded;  masculinity  vs.  femininity,  the  degree  to 
which  men  and  women  within  a  culture  differ  in  their  respective  roles;  uncertainty  avoidance, 
which  reflects  how  well  individuals  handle  unexpected  and  ambiguous  situations;  and  time 
orientation,  the  degree  that  a  culture  does  or  does  not  embrace  a  long-term  perspective  and  a 
respect  for  traditions.  The  scores  of  each  nation  on  the  Hofstede  dimensions  are  represented  in 
Figure  5. 
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Uncertainty  Avoidance  Index  Long-Term  Orientation 

Figure  5:  Map  of  national  scores  on  Flofstede’s  five  cultural  dimensions 
Note;  Darker  shading  indieates  higher  seores.  Blank  areas  indieate  missing  data.  This  map  was 
generated  using  the  ManyEyes  web  service'”  using  a  data  set  published  by  Hoftstede”. 

Related  work  by  Nisbett,  2003)  and  others  has  introduced  the  importance  of  honor,  or  the  degree 
to  which  society  sanctions  the  use  of  violence  as  a  justifiable  tactic  for  resolving  affronts  to  one’s 
character.  Under  Hofstede’s,  Arab  culture  is  characterized  by  high  power  distance  (meaning 
wide  disparities  of  equality  among  social  classes  and  great  deference  to  status  individuals),  and 
high  uncertainty  avoidance  (in  which  daily  interactions  are  governed  by  societal  rules  that  lower 
uncertainty).  Beaubien,  Knott,  Orvis,  and  Freeman  (2008)  developed  descriptions  of  situations 
that  tested  the  standing  of  respondents  on  most  of  the  dimensions,  above,  and  found  that  unique 
linear  combinations  of  these  dimensions  reliably  predicted  the  responsiveness  of  members  of  one 
Asian  culture  to  strategies  typically  used  in  advertising  and  psychological  operations  . 

Traditional  demographics  analyses  are  often  used  to  profde  cultures  for  purposes  as  diverse  as 
market  analysis,  political  forecasting,  and  psychological  operations,  in  which  decisions  concern 
what  style  of  communications  -  rational  explanations,  visionary  statements,  testimonials,  etc.  - 
will  most  effectively  convey  a  message.  Demographic  analysis  typically  considers  data 
concerning  income,  age,  sex,  working  status,  language,  position  in  the  household  (father, 
daughter,  etc.),  marital  status,  and  religion.  These  variables  more  or  less  reliably  predict 
purchasing  behaviors,  voting,  and  other  behaviors.  Note,  however,  that  demographic  methods  are 
typically  within  a  culture,  rather  than  between  cultures.  It  is  not  entirely  clear  whether  traditional 
demographics  reliably  predict  institutional  behaviors  or  define  its  culture. 


***  ManyEyes  is  available  at:  http://serviees.alphaworks.ibm.eom/manveves/view/SLinXQsOtha6b6FrBV6bQ2~ 

"  The  Flofstede  data  set  is  published  by  the  author  at:  http://www.geert-hofstede.oom/hofstede  dimensions.php 
The  goal  of  PSYOP  is  to  always  tell  the  truth,  as  evideneed  by  the  motto  of  COL  James  Treadwell,  Commander  of 
the  4th  Psyehologieal  Operations  Group:  “The  truth  is  the  best  propaganda.”  This  strategy  helps  ensure  the 
eredibility  of  influenee  eampaigns  (U.S.  Joint  Forees  Command,  2003). 
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Appendix  B:  Trident  Warrior  Technologies  in  HI08.1 

Two  collaboration  technologies  were  explored  in  the  part  of  TWOS  focused  on  law  enforcement 
(HI08.1).  These  technologies  were  the  Law  Enforcement  Information  Exchange  (EInX)  and  the 
Eorce  Proteetion  Portal  (EPP). 

•  EInX  is  a  database  of  law  enforcement  records  from  several  partieipating  regions  of  the 
United  States.  EInX  presents  the  user  with  aceess  to  pawn  shop  data;  a  rapid,  tactieal 
search  capability  typically  used  by  poliee  during  arrests;  an  analytic  search  capability  that 
provides  Boolean  seareh,  seareh  of  free  text  (narrative)  fields,  and  link  analysis;  a 
personalized  home  page;  and  system  administration  functions.  A  search  on  EInX  can 
return  a  suspeet’s  name,  date  of  birth,  aliases,  eriminal  incidents,  role  in  those  incidents 
(witness,  cited),  vehicles  involved,  weapons  involved,  ineident  type,  a  narrative  (e.g., 
pocket  litter),  photo,  warrant  information,  contact  information  for  the  suspect,  and  contact 
information  for  the  issuing  law  enforeement  agency.  Link  analysis  reports  graphically 
represent  the  relationships  between  many  of  these  entities  (e.g.,  suspects  to  addresses  to 
vehicles).  The  quality  of  these  reports  varies  somewhat  due  to  uncorrelated  data  (e.g., 
separate  records  for  John  Hartwick,  John  J.  Hartwick,  and  Johnny  Hartwik  may  concern 
the  same  person),  spurious  correlations  (by  whieh  innocent  individuals  are  coineidentally 
associated  with  suspects).  The  utility  of  the  reports  is  in  part  a  function  of  their  volume;  a 
search  against  the  current  one  million  records  can  return  a  very  large  number  of  hits. 

Users  generally  may  not  use  EInX  reeords  as  the  basis  for  warrants  or  arrests,  but  must 
contaet  the  issuing  agency  for  the  source  data.  EInX  legal  records  may  not  be  transferred 
to  the  military.  These  and  other  polieies  are  eurrently  being  standardized  aeross  the  EInX 
user  community.  In  this  exercise,  EInX  was  populated  with  actual,  but  anonymized 
criminal  records. 

•  EPP  is  an  unclassified,  password  protected  internet  portal  that  provides  file  sharing,  ehat, 
email.  EPP  presents  users  with  access  to  news,  upcoming  events,  a  document  repository, 
sites  of  interest  (DHS,  Department  of  State,  CNN,  etc.),  and  the  NCIS  portal.  The  latter 
provides  email,  a  document  repository,  ehat,  a  ealendar,  and  eurrent  briefings.  NCIS 
participants  and  observers  in  TWOS  exercised  the  EPP  portal  to  access  doeuments 
concerning  seenario  events,  and  to  communicate  with  eaeh  other  and  the  controller.  All 
users  had  aceess  to  all  doeuments  on  the  portal.  (In  actual  use,  there  may  be  legal  or 
regional  policy  constraints  on  information  access  and  dissemination).  EPP  is  designed  for 
operational  use,  and  not  for  experimentation.  Thus,  it  does  not  support  observers  with 
features  sueh  as  email  logging. 

The  scenarios  that  exercised  EInX  and  EPP  in  this  part  of  Trident  Warrior  consisted  of  base 
materials  and  a  four-day  stream  of  injects  that  required  participants  to  detect  several  threats: 
Scenario  la:  A  planned  anthrax  attack  against  Oahu 
Scenario  lb:  A  planned  anthrax  attaek  against  Seattle 
Scenario  2:  A  planned  radiological  attack  against  a  Hawaiian  sonar  range 
Participants  in  this  element  of  Trident  Warrior  field  offices  of  the  Naval  Criminal  Investigative 
Service  (NCIS)  in  Bahrain,  Dubai,  Hawaii,  Japan,  Seattle  and  Singapore;  the  Joint  Terrorism 
Task  Eoree,  Third  Eleet  MOC,  the  British  Royal  Navy,  and  the  New  Zealand  Navy.  In  addition, 
data  were  collected  in  an  observation  of  a  VTC  between  several  of  these  organizations  and  the 
Multiple  Threats  Alert  Center. 
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Appendix  C:  HI08  Data 


Data  collected  at  HI08  are  available  in  sanitized  form  (i.e.,  without  personally  identifying 
information)  with  the  permission  of  OSD  NIL  For  further  information,  please  contact  Jared 
Freeman,  freeman@aptima.com,  202-842-1548  x316. 
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Appendix  D:  Controller  communications  in  HI08.1 


In  addition  to  the  interviews  reported  in  the  body  of  this  report,  we  examined  a  sample  of  email 
between  participants  and  the  controller  in  HI08. 1  in  order  to  understand  the  level  of 
participation  and  the  types  of  interaction  between  participants  and  experiment  organizers.  We 
found  several  classes  of  communication: 

•  Technical  support  communications  provided  users  with  instructions  about  where  to  put 
files,  how  to  use  the  systems,  how  to  handle  different  file  formats,  and  so  forth. 

•  Logistical  communications  concerned  coordination  between  controllers,  how  to  reach 
controllers,  etc. 

•  Scenario  event  communications  provided  new  injects,  copies  of  injects,  background 
information,  responses  to  requests  for  information  (e.g.,  summaries  of  requested  (but 
unavailable)  reports),  responses  to  participants’  planned  actions  (e.g.,  “When  you  call  the 
Bahraini  Minister  of  the  Interior,  you  will  learn  that. . .”),  and  corrections  of  participant 
misunderstandings . 

•  Participant  activity  reports  documented  what  participants  had  done  and  what  they 
planned  to  do  within  or  outside  the  artificial  confines  of  the  scenario.  Only  one 
participant  in  the  sample  of  '-SO  emails  provided  detailed  accounts  of  these  activities. 


No  email  between  partieipants  was  available  for  analysis. 
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Appendix  E:  Notes  Towards  Experimentation 


Experiment  Themes 

Several  speeific  themes  for  HI09  experimentation  arose  in  discussions  among  members  of  the 
research  team: 

•  Optimizing  Human  Interoperability  Performance  in  Law  Enforcement  Enterprises  which 
Eeverage  Investigative  Data  Sharing  Systems  (e.g.,  LInX) 

•  The  Role  and  Contribution  of  HI  in  the  Developing,  Establishing  and  Advancing 
Common  Operating  Pictures 

•  Aligning  Distinct  Different  Organization  Cultures  through  HI  Principles,  Standards  and 
Practices  for  Improved  Enterprise  Performance,  Study  One:  The  Military  and  Embedded 
and  External  Eaw  Enforcement  (this  could  be  the  first  in  a  series,  which  could  go  on  and 
on) 

•  The  Contribution  of  HI  to  Senior  Policy  Eormulation  and  Decision  Making  (contact  Neil 
Palmer,  UK  MOD) 

•  Agency  Ideology  and  Culture  and  Its  Influences  on  Establishing  and  Maintaining  Peer 
External  Organizational  and  Individual  Relationships  (JTTE  as  one  experimental 
environment) 

•  Trust,  Relationships  and  Enterprise  Effectiveness  in  Multiagency  Constructs  (established 
as  in  JTTEs  for  law  enforcement,  or  task  forces — various  military-  non-military 
constructs;  hastily  formed  and  established  or  mandated  and  forming) 

•  Influence  of  Personnel  and  Career  Practices  on  Enterprise  Performance  in  Agencies 
Dependent  on  Specialized  Relationships,  Experience  and  Expertise 

•  Role  of  Relationships  in  DOD-Non-DOD  Engagements:  The  NCAG  Model  (can  identify 
others  for  study  too  to  broaden  the  effort) 

•  Comparison  of  International  and  USN  NCAG  Operations:  The  HI  Component 

•  HI  in  Crisis  and  Non-Crisis  Environments,  Strategic,  Operational  and  Tactical 
Considerations  In  Established  and  Hastily  Eormed  Multiagency  Constructs 


Experimental  Participants 

HI08.1  successfully  engaged  military  law  enforcement.  HI09  should  engage  civilian  entities,  as 
well. 

Recommendation:  An  NCIS  analyst  s  recommended  that  future  experiments  engage  a  major 
police  department  with  command-and-control  capabilities  such  as  the  NYPD,  DHS  entities  such 
as  the  FBI,  and  European  law  enforcement  entities  such  as  EUROPOL. 


Experimental  Materials  &  Process 

A  data  collection  instrument  was  defined  to  elicit,  in  interviews  and  observations,  data 
concerning  policy  barriers,  inter-  and  intra-organizational  processes  and  structure,  and  other 
matters.  The  draft  instrument  is  in  Appendix  E:  Data  Collection  Instruments 
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Recommendation:  Review,  revise,  and  use  the  draft  data  collection  instrument  in  HI09. 

The  analysis  of  controller-participant  communications  identified  several  classes  of 
communications  for  which  controllers  might  prepare  in  advance  of  the  experiment. 

Recommendation:  To  prepare  for  their  technical  support  role,  controllers  should  conduct  at 
least  one  pilot  study  to  identify  the  types  of  technical  support  questions  that  may  arise  and  the 
types  of  workarounds  that  may  be  needed. 

Recommendation:  To  prepare  for  their  technical  support  role,  controllers  should  conduct  one 
tabletop  exercise  using  the  experimental  materials  to  determine  what  information  or  documents 
participants  may  request  from  whom,  how  to  structure  fileshares,  how  to  conduct  chats  (e.g., 
how  to  schedule  them  across  time  zones),  and  so  forth. 

Recommendation:  White  cell  and  controllers  should  map  out  the  potential  network  of  contacts 
with  whom  participants  might  try  to  communicate  during  the  scenario,  specify  the  questions  that 
might  be  asked,  and  sketch  out  provisional  answers  to  deliver  in  response. 

Recommendation:  If  it  is  not  possible  to  monitor  participant  activity,  controllers  should 
encourage  participants  to  document  the  activities  they  did  execute  (e.g.,  “Issued  RFI 
concerning...  ”)  and  those  they  plan  to  execute  (e.g.,  “Will  call  the  Bahrain  Ministry  of  the 
Interior  to  ask...  ”). 

Experimental  Venue 

A  fundamental  concern  in  inter-agency  experiments  is  where  participants  should  be  as  they 
execute  the  experiment. 

Participants  who  convene  at  a  central  site  lose  access  to  some  of  the  human  and  material 
resources  on  which  they  depend,  but  they  are  at  less  risk  of  distraction,  they  have  the  technical 
support  they  may  need  to  use  experimental  technologies  effectively,  and  they  are  readily 
accessible  to  observers. 

Participants  working  from  their  own  workplaces  have  access  to  the  human,  technical,  and 
document  resources  that  enable  them  to  respond  to  scenario  events  in  a  natural  way.  However, 
they  are  inevitably  distracted  by  real-world  tasking,  they  may  have  difficulty  using  new 
technologies  or  procedures  being  studied  in  the  experiment,  and  it  can  be  more  difficult  to 
observe  their  behaviors  or  elicit  their  opinions. 

Distributed  participants  are  inevitable  in  any  richly  defined  inter-agency  experiment.  Methods 
are  needed  to  engage  distributed  participants  more  fully  in  the  experiment. 

Recommendation:  Place  a  trusted  agent  working  at  each,  distributed  site  to  win  fuller 
engagement  from  participants  there. 
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Recommendation:  Engage  distributed  participants  in  pre-experiment  planning,  in-experiment 
hot  washes,  and  post-experiment  lessons  learned  events  to  develop  a  community  of  interest  and 
raise  their  level  of  involvement  in  the  experiment.  In  these  sessions,  emphasize  the  institutional 
value  of  the  experiment  (e.g.,  fulfilling  an  ofificial  requirement),  and  the  professional  benefits  to 
participants:  learning  new  technologies,  building  relationships  with  others  in  the  field,  fulfilling 
goals  for  performance  review,  or  intellectual  challenge. 


Organizational  Structures 

A  variety  of  organizational  structures  for  HI  were  identified  in  HI08.1. 

Recommendation:  HI09  should  sample  the  variety  of  structures  used  to  build  HI:  co-location  of 
agency  representatives,  collaboration  through  trusted  parties,  and  distributed  collaboration 
through  technology. 


Organizational  Culture 

Law  enforcement  and  military  organizations  exhibited  important  cultural  differences  in 
HI2008.1.  The  next  HI  should  exploit  such  differences. 

Recommendation:  HI09  should  assess  the  cultural  attributes  of  participating  organizations 
(e.g.,  using  Hof stede’s  (1980)  framework)  and  should  design  scenarios  that  exploit  natural 
interactions  between  entities  with  conflicting  cultures. 


Organizational  Process 

Law  enforcement  and  the  military  differ  in  the  processes  they  employ  for  developing 
relationships  between  entities. 

Recommendation:  HI09  should  assess  the  formality  and  effectiveness  of procedures  for 
institutionalizing  productive  relationships  between  entities. 


Information  systems 

Observers  of  HI08.1  saw  successful  use  of  LInX  for  searches  of  law  enforcement  data,  and  FPP 
for  relaying  scenario  injects  and  communication  with  the  experiment  controller.  Lessons  learned 
concerning  the  FPP  are  particularly  relevant  to  future  experiments,  which  will  use  collaboration 
technologies.  In  general,  these  technologies  or  technologies  (“wrappers”)  that  incorporate  them 
should  provide  services  for  experiment  observers,  not  just  participants. 

Recommendation:  Ensure  that  portal-based  email  systems  copy  every  email  between 
participants  so  that  these  messages  are  also  delivered  to  controllers  and  observers.  Ideally, 
participant  email  addresses  will  include  their  locations  and/or  roles. 

Recommendation:  Ensure  that  chat  systems  log  all  communications  for  inspection  by  observers. 
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Recommendation:  Ensure  that  file  sharing  system  log  file  activity  for  analysis  (e.g.,  to  determine 
who  did  and  did  not  read  or  write  key  files).  It  may  be  useful  to  push  these  logs  like  RSS  feeds  to 
observers  and  controllers. 

Recommendation:  Ensure  that  phone  systems  records  voice  communications  and  metadata, 
minimally  participant  identities,  date,  time,  and  call  duration  for  each  participant  for  each  call 
(point-to-point  or  teleconference).  Providing  VOIP  telephone  capability  through  portals  may 
facilitate  this. 

Several  minor  teehnieal  glitehes  eonveyed  useful  lessons. 

Recommendation:  Ensure  that  participants  have  access  to  the  infrastructure  required  to  use  new 
technology:  unclassified  network  ports,  utilities  for  opening  required  files  (e.g.,  .pdf’s),  etc. 

Providing  the  human  infrastrueture  for  new  technologies  is  an  important  aspect  of  technology 
insertion. 

Recommendation:  Technology  evaluations  should  assess  the  staffing  requirements  of  new  tools, 
the  utility  of  training  for  performing  locally  critical  tasks,  and  the  plans  for  communities  of 
interest  to  govern  the  use  of  new  technology. 
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Appendix  F:  Data  Collection  Instruments 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: _ 

Instructions  to  HI08  Data  Collectors 

Please  read  the  instructions  for  researchers  to  familiarize  yourself  with  the  objectives  of 
the  HI  experiment  objectives. 

Please  use  the  Briefing  to  Participants  if  you  must  provide  a  brief. 

Collect  data  at  major  scenario  events  using  the  questionnaire,  log,  and  other  forms. 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: 


Overview  for  HI08  Data  Collectors 

HI08  is  an  experiment  designed  to  evaluate  human  interoperability  in  a  complex 
environment  -  an  inter-agency  maritime  security  experiment  by  assessing 

•  Policies  concerning  information  sharing 

•  Processes  &  structures  within  agencies  that  support  collaboration 

•  Processes  &  structures  between  agencies  that  implement  collaboration  (e.g., 
cross-briefings,  working  groups) 

•  Response  speed  within  and  between  agencies 


Organization  #1 

Organization  #2 

Policies  for  HIE 

Policies  for  HIE 

Internal 
Procedures 
&  Structures 
for  HIE 

External 
Procedures 
&  Structures 

Internal 
Procedures 
&  Structures 
for  HIE 

for  HIE 


Figure  6:  HI08  explores  the  policies,  procedures,  aud  structures  (iu  italics)  that  euahle  HI. 

The  products  of  this  experiment  and  future  work  will  be; 

•  A  process  for  planning  and  conducting  HI09 

•  Gaps  and  opportunities  for  enhancing  collaboration  and  coordination. 

•  New  organizational  structures  and  processes  that  enable  disparate  organizations  to 
collaborate  across  legacy  shared  or  ad-hoc  boundaries 

•  Reliable  methods  for  measuring  and  managing  human  interoperability  in 
organizations 

HI08  takes  place  16-27  June  2008  at  various  locations  as  part  of  Trident  Warrior  2008. 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Partieipant  Role(s)  /  Loeation 

Vignette  /  Event: 


Participating  data  collectors  in  this  experiment  are; 


Person 

Date 

Task 

Email 

Cell 

Tami 

Smith 

23-27 

HI  data 
eollection 

tami .  smith@navy  .mil 

703-362- 

5940 

Randy 

Murch 

16-20, 

23-27 

HI  data 
eollection 

rmurch@vt.edu 

703-731- 

2712 

Jared 

Freeman 

16-27 

HI  data 
collection 

freeman@aptima.eom 

202-262- 

3030 

Alenka 

Brown 

22-27 

HI  data 
collection 

stefania.brown- 

vanhoozer@osd.mil 

571-309- 
4951  /703- 
607-0740 

Hal  Moore 

23-27 

HI  white  cell 

hal.moore@northcom.mil 

(719)  554- 
8045 

Bob  Miley 

TWOS  lead  for 
NCIS 

Tbd 

202-433- 

9103 

Shelley 

Gallup 

16-27 

TWOS 

management 

SDgalluD@nDS.edu 

831-594- 

0609 

Dan 

Dunaway 

16-27 

TWOS 

management 

ddunaway@systeehnologies.eom 

tbd 

Kevin 

Kurtz 

TWOS 

manager 

kkurtz@systechnologies.com 

757-651- 

4800 

The  instruments  below  are  designed  for  use  by  the  data  eolleetors.  Please  print  several 
copies  and  take  notes  on  these  forms,  or  take  notes  as  you  wish  while  keying  the  answers 
to  the  question  numbers  listed  below.  Please  administer  these  forms  at  every  significant 
experimental  event  or  seenario  as  defined  in  the  TWOS  schedule.  Amend  the  forms  with 
new  items  as  you  see  fit.  Please  write  neatly  or  transeribe  your  notes  to  eleetronic  form. 
Keep  all  notes  unclassified. 

Return  all  materials  to  Jared  by  1  July  2008.  He  will  send  Fed  Ex  to  you  upon  request. 
Contaet  Jared  at:  freeman@aptima.eom:  offiee;  202-842-1548,  x316;  eell:  202-262-3030; 
fax:  202-842-2630;  Aptima,  1726  M  Street,  NW,  #900,  Washington,  DC  20036. 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: 


Form  #1:  Briefing  to  Participants 

The  Office  of  Secretary  of  Defense  is  sponsoring  an  experiment  to  examine  how 
organizations  interact  over  Maritime  Domain  Awareness  events.  We  call  this  experiment 
Human  Interoperability  2008.  This  experiment  “piggybacks”  on  Trident  Warrior  2008, 
which  you  are  participating  now. 

We  are  not  assessing  your  performance  in  HI08.  We  are  not  assessing  the  performance  of 
your  organization.  We  are  studying  the  interface  between  organizations,  specifically  within 
DoD  and  between  DoD  and  DoJ,  DHS  and  the  Intel  Community. 

OSD  will  issue  a  report  from  this  experiment  that  draws  some  initial  lessons  and  specifies 
the  design  of  a  larger  HI  experiment  in  2009.  Future  products  of  HI  will  help  organizations 
to  design  their  policies,  processes,  protocols,  organizational  structures,  standards  and 
technologies  in  ways  that  make  MDA  and  other  complex  organizational  enterprise 
responses  more  effective,  efficient,  timely  and  responsive. 

This  is  the  support  we  need  from  you: 

•  During  parts  of  TW08; 

o  A  data  collector  will  sit  with  you  to  observe  and  ask  you  questions, 
o  WeTl  request  your  job  title  and  organization.  (We  don’t  need  your  name), 
o  We'll  observe  your  response  to  events  in  TW08,  and  we  will  interview  you 
concerning  these  events. 

•  Please  be  prepared  to  introduce  the  data  collector  to  other  knowledgeable, 
appropriately  placed  experts  in  your  organization.  We'd  like  to  gather  their  thoughts 
about  the  type  of  events,  processes  and  opportunities  for  improvements  to  human 
systems  performance  that  will  be  encountered  in  this  exercise. 

•  Please  play  by  the  rules  of  TW08.  Our  presence  is  not  meant  to  change  your 
mission  or  actions  in  TW08. 

Do  you  have  any  questions? 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: 


Form  #2:  Participant  Demographics 
Instructions 

For  each  participant  observed  or  interviewed,  gather  the  information  below.  Develop  and 
document  new  questions  as  you  see  fit. 

Demographics _ 

2.1  Operator  TWOS  PIN  #  (4  digits  chosen  by  operator) 

2.2  First  letter  of  Operator’s  last  name; 

2.3  Rank  /  Rate 

2.4  Organization  /  Command; 

2.5  Position/Role  Title; 

2.6  Length  of  Time  in  Position/Role; 

2.7  Description  of  responsibilities: 

2.8  Length  of  Time  in  MDA  Missions; 

2.9  List  or  network  of  superiors,  peers,  subordinates  by  role  and  function. 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: 


Form  #3:  Interview  Questionnaire 
Instructions 

For  each  major  event  or  scenario,  ask  the  participant(s)  the  Questions  (below).  Record  the 
question  numbers  and  answers  in  Observer  Notes  table.  Develop  and  document  new 
questions  as  you  see  fit. 


Questions 


Q# 

Item 

Objective 

3.1 

How  does  your  organization  decide  to  accept  this  task  or 
respond  to  an  event  or  message  like  this? 

Objective  v2;  Process 
between  agencies  (Item; 
14.01  Interface: 
process) 

3.2 

What  is  an  acceptable  range  of  time  (minimum,  average, 
and  maximum)  to  respond  to  an  event  such  as  this? 

Objective  v2;  Process 
between  agencies  (Item; 
5.01  Interface;  n.a.) 

3.3 

How  do  you  process  this  task  internally?  Who  does 
what? 

Objective  v2:  Process 
between  agencies  (Item; 
14.02  Interface; 
process) 

3.4 

Is  there  a  format  or  protocol  for  sharing  information 
about  this  event  between  agencies?  If  yes,  name  it. 

Objective  v2;  Process 
between  agencies  (Item; 

1. 01  Interface;  data) 

3.5 

Does  the  format  convey  the  information  that  is  necessary 
and  sufficient  for  analysis  or  decision  making  in  your 
(receiving)  organization?  If  no,  please  comment. 

Objective  v2;  Process 
between  agencies  (Item; 
1.02  Interface:  data) 

3.6 

What  interactions  with  other  agencies  will  you  engage  in 
to  respond  to  this  event?  How  long  can  you  afford  to  wait 
(i.e.,  the  maximum)  for  data  or  decisions  from  the  other 
agencies? 

Objective  v2;  Process 
between  agencies  (Item; 
6.02  Interface;  process) 

3.7 

Are  interagency  teams/collaborations  defined  before  the 
event?  If  yes,  name  them. 

Objective  v2;  Process 
between  agencies  (Item; 
1.04  Interface; 
structure) 

3.8 

How  do  you  ensure  that  responses  or  tasks  are  ready  to 
hand  off  to  other  agencies? 

Objective  v2;  Process 
between  agencies  (Item; 
14.03  Interface: 
process) 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: 


Q# 

Item 

Objective 

3.9 

Is  there  a  named  process  for  sharing  information 
about  this  event  between  agencies  (e.g.,  a 
protocol,  meeting,  COP)?  If  yes,  name  it. 

Objective  v2;  Process  between 
agencies  (Item;  1.03  Interface; 
process) 

3.11 

Are  there  policies  that  enable  interaction 
between  agencies  in  this  event?  What  are  they? 

Objective  v2:  Process  between 
agencies  (Item;  9.01  Interface: 
policy) 

3.12 

What  interactions  in  this  event  are  not  governed 
by  policy? 

Objective  v2;  Process  between 
agencies  (Item:  1 0.0 1  Interface: 
policy) 

3.13 

Are  these  policies  ever  suspended  to  facilitate 
interagency  interaction?  Under  what  conditions? 

Objective  v2:  Process  between 
agencies  (Item:  1 1. 01  Interface: 
policy) 

3.14 

Do  the  policies  of  other  agencies  conflict  with 
these  policies? 

Objective  v2:  Process  between 
agencies  (Item:  13.01  Interface: 
policy) 

3.15 

What  inhibits  effective  performance  on  this 
event? 

Objective  v2:  Process  between 
agencies  (Item;  3.01  Interface; 
policy,  structure,  process,  data, 
culture) 

3.16 

Is  performance  on  this  event  handicapped  by  the 
structure  of  your  organization;  insufficient 
number  of  staff,  missing  roles,  redundant  roles, 
unnecessary  roles? 

Objective  v2:  Process  between 
agencies  (Item;  3.02  Interface; 
structure) 

3.17 

Is  performance  on  this  event  handicapped  by 
missing  or  awkward  processes? 

Objective  v2:  Process  between 
agencies  (Item;  3.03  Interface; 
process) 

3.18 

Is  performance  on  this  task  handicapped  by 
difficulty  getting  specific  data  or  information? 

Objective  v2;  Process  between 
agencies  (Item;  3.04  Interface: 
data) 

3.19 

Is  performance  on  this  task  handicapped  by 
cultural  or  personality  conflicts  within  or 
between  organizations? 

Objective  v2:  Process  between 
agencies  (Item;  3.05  Interface; 
culture) 

3.20 

Are  there  policies  that  inhibit  interaction 
between  agencies  in  this  event?  What  are  they? 
How? 

Objective  v2;  Process  between 
agencies  (Item;  9.02  Interface: 
policy) 
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Observer  Notes 


0# 

Notes 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: 


Form  #4:  Observation  Log 
Instructions 

For  each  major  event  or  scenario,  record  significant  events  pertaining  to  each  of  the  items 
using  the  Log  and  Log  Addendum.  Develop  and  document  new  items  as  you  see  fit. 


Log 


# 

Item 

Objective 

4.1 

What  data  or  decisions  are  received  at  the  beginning  of 
this  event?  What  is  the  format  or  protocol?  In  what 
medium?  From  what  entity? 

Objective;  Process 
between  agencies 

4.2 

When  did  the  event  arrive  at  the  agency? 

Objective: 

Responsiveness 

4.3 

When  did  this  entity  or  agency  accept  responsibility  for  or 
first  make  a  significant  response  to  this  event?  Describe 
the  response. 

Objective; 

Responsiveness 

4.4 

What  is  the  name  (if  any)  of  the  organizational  process 
that  is  invoked  in  response  to  this  event?  (“Sharing 
processes”) 

Objective;  Process 
between  agencies 

49 


Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: _ 


# 

Item 

Objective 

4.5 

What  teams/collaborations  participate  in  the  event? 

(“Sharing  participants/venues”) 

Objective:  Process 
between  agencies 

4.6 

What  data  or  decisions  are  requested  while  processing  this 
event?  By  whom  and  of  whom?  What  is  the  format  or 
protocol?  In  what  medium?  (“Sharing  behaviors”) 

Objective;  Process 
between  agencies 

4.7 

What  policies  (if  any)  are  cited  during  task  execution? 

Objective:  Policies 

4.8 

What  tasks  are  performed  in  response  to  this  event? 

(“Sharing  behaviors”) 

Objective:  Process 
within  agencies 

4.9 

When  did  the  agency  issue  a  response  or  decision 
concerning  the  event? 

Objective: 

Responsiveness 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Partieipant  Role(s)  /  Loeation 

Vignette  /  Event: 


Log  Addendum 


# 

Notes 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: 


Form  #5:  Workplace  Layout 
Instructions 

For  each  physical  space  in  which  you  observe  operations,  sketch  the  configuration  and 
indicate  the  roles  and  organizations  of  each  operator  /  decision  maker. 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: 


Form  #6:  Social  Network 
Instructions 

For  each  organization  you  observe,  sketch  the  social  network  within  and  between 
organizations  in  which  organizations  and  roles  are  nodes,  and  directed  arcs  (arrows)  are 
relationships.  Please  label  the  arcs  with  terms  that  characterize  the  function  the 
relationship  serves  (e.g.,  “Role  #1  is  liaison  to  Organization  #2”). 
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Human  Interoperability  2008 

Data  collector;  Date/Time; 

Observed  Organization  / 

Participant  Role(s)  /  Location 

Vignette  /  Event: 


Form  #7:  HI09  Recommendations 
Instructions 

Record  below  the  table  your  ideas  for  an  efficient,  effective  HI09  experiment. 


Q# 

Item 

7.1 

Your  ideas  for  HI09  research  objectives 

7.2 

Your  ideas  for  HI09  research  measures 

7.3 

Your  ideas  for  HI09  research  participants  (organizations) 

7.4 

Your  ideas  for  HI09  scenario  events  &  materials 

7.5 

Your  ideas  for  HI09  research  methods 

7.6 

Your  ideas  for  HI09  analyses  &  reporting 

# 

Notes 
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1 .  Defense  Teehnical  Information  Center 
8725  John  J.  Kingman  Rd.,  STE  0944 
Ft.  Belvoir,  VA  22060-6218 

2.  The  Naval  Postgraduate  Sehool 
1411  Cunningham  Road,  Code  06IS 
Monterey,  CA  93943 

3.  Dudley  Knox  Library,  Code  013 
Naval  Postgraduate  Sehool 
Monterey,  CA  93943-5100 

4.  Aptima,  Ine. 

1726  M  Street,  NW,  Suite  900 
Washington,  DC  20036 

5.  Virginia  Teeh 

1101  King  Street,  Suite  610 
Alexandria,  VA  22314 

6.  NORTHCOM 

250  Vandenberg  Street,  Suite  BO  16 
Peterson  AFB,  CO  80914-3821 

7.  Department  of  Homeland  Security 
1120  Vermont  Avenue 
Washington,  DC  20005 

8.  Dr.  Alenka  Brown- VanHoozer  (Via  e-mail) 

9.  Dr.  Shelley  Gallup  (Via  e-mail) 

10.  Dr.  Jared  Freeman  (Via  e-mail) 

1 1 .  Dr.  Randall  Murch  (Via  e-mail) 

12.  Tami  Smith  (Via  e-mail) 

13.  Hal  Moore  (Via  e-mail) 

14.  Art  Glynn  (Via  e-mail) 


15.  Jeff  Chilton  (Via  e-mail) 


